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ABSTRACT 

The document is part of the final report on Project 
STEEL (Special Teacher Education and Evaluation Laboratory) intended 
to extend the utilization of technology in the training of preservice 
special education teachers. This volume focuses on the first of four 
project objectives, the development and implementation of a 
microcomputer-based assessment system for preservice special 
education teachers. Activities of Year 1 included data gathering on 
teacher behaviors, software development, observer training and field 
testing, and trainee feedback. Year 2 activities involved revision of 
the microcomputer-based observation system (MBOS), Project STEEL 
observation system field t'^sts, application to the undergraduate 
program, use by graduate supervisors, and students' reactions to the 
system. Year 3 activities included evaluation of the preservice 
teacher group data, and an effectiveness study. This study is 
presented in its entirety. Preservice special education students 
(N=27) in the multiple baseline stud^ received information on the 
concept of Academic Learning Time (ALT) and the relationship between 
this information and MBOS. Feer'back was provided in the form of 
either traditional field notes or computer-based feedback. Results 
demonstrated that the computer-based feedback significantly increased 
rates of effective teaching behavior compared to field note feedback. 
(DB) 
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Project STEEL Final Report 



OVERVIEW 

This report describes developed products, research, and evaluation 
regarding the computer-based Special Teacher Education and Evaluation 
Laboratory (STEEL) at the Center for Innovation in Teaching the 
Handicapped (CITH), School of Education, Indiana University, 
Bloomington. Four major goals were achieved in Project STEEL: 

I. Development, implementation, and evaluation of a microcornputer- 
based observation system for codification, storage, and summarization of 
special education trainees* classroom teaching performances 
(STEEL/MBOS); 

II. Development, field testing^ and evaluation of computer 
literacy training procedures and materials for preservice and inservice 
special education teachers (STEEL/COLT); 

III. Development, implementation, and evaluation of a computer- 
based testing system for assessing teacher knowledge (STEEL/CBTS ) ; and 

IV. Development and preliminary evaluation of a computer-based 
information raana-enent system for storing and retrieving data on special 
education teachers* performances during their preservice training 
program (STEEL/ IMS). 

Comprehensive descriptions of each of these major accomplishmt^nt s 
are provided in four separately bound reports (Volumes I through IV, 
respectively). A fifth separately bound report contains the executive 
summary of Volumes I through IV, and should be read first* 

This document contains Volume I only. 
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Development and Evaluation of the STEEL/MBOS 
Year 1 Activities 

During the first year of the Project, priority was devoted to the 
development and field testing of the STEEL Observation System. 
Observation syste-aS utilized in the Beginning Teacher Evaluation Studies 
(Fisher et al., 1978) and the Academic Learning Time research conducted 
<iL « x:7w^/ were reviewed. A series or consecutive 

meetings were held by the STEEL staff, and a prototype observation 
system was developed. Several revisions of ^his system were undertaken 
prior to its being field tested. These revisions were conducted in a 
series of meetings with the CITH staff, graduate student supervisors who 
were to Use the system in the field, and key special education faculty 
members. When consensus had been reached regarding the formative version 
of the instrument, five supervisors were trained to use the system with 
the Epson HX-20 portable aiicrocomputer and field tested it in the 
classrooms. 

In the initial year of the project, field supervisors gathered 
data on the teaching behavior of fifty-four undergradu^^te teacher 
trainees over a period of eight weeks. This perioc^rved as the pilot 
phase in developing the observacion system. Following field testing, 
graduate supervisors were thoroughly debriefed. The> shared their 
perceptions of the system and consumer satisfaction information. Based 
on this varied feedback, the observation instrument was revised again 
over the following summer. The formative version of this system involved 
a 59-code teacher and student behavior observation procedure and a 
95-item checklist of qualitative indicators to be used in evaluating 
student trainees* classroom performances (see Attachment A). 
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ERIC 



STEEL OBSERVATION SYSTEM 



IDENTIFICATION INFORMATION 



DATE: 



OBSERVER: 



TEACHER OBSERVED: 



- J, 
ID # 

ID # 



CATEGORIES FOR CODING OF CLASSROOM, TEACHER, AND STUDENT VARIABLES 



1. Classification: Group S tructure (in relation to the observed teacher*s focus) 

Categories: 10 Null 

11 Whole group responsibil ity 

12 Partial group responsibility 

13 Partial group responsibility with additional monitoring 

14 Ind ividual respons ibility 

15 Individual responsibility with additional monitoring 
19 Can't tell 



2. Classification: 
Categories : 



Class if ication: 
Categories : 



Class if ication : 
Categories : 



5. Classification: 
Categories : 



Educational Activity 

20 Null 

21 Act ive Instruction 

22 Passive Instruction 

23 Trans it ion/Procedural 

29 Can't tell 

Teacher Instructional Beha viors 

30 Null 

31 Preparat ion/Administrat ive Duties 

32 Observational Monitoring 

33 Struct uring/Direct ing 

34 Explanation/Quest ioning - Planned 

35 Explanation/Questioning Need 

36 Evaluat ive Feedback 

37 Task Engage lent Feedback 

38 Behavioral Feedback 

39 Can't tell 

Student(s) behaviors (Target gtoup) 

40 Null 

41 Engaged - Active 

42 Engaged - Passive 

43 Non-Engaged - Active 

44 Non-Engaged - Passive 

49 Can't tell 

Student(s) Behaviors (Monitored group) 

50 Null 

51 Engaged - Active 

52 Engaged - Passive 

53 Non-Engaged - Active 

54 Non-Engaged - Passive 
59 Can't tell 
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STEEL OBSERVATION SYSTEM 
OPERATIONAL GUIDELINES 



GROUP STRUCTURE: This classification describes the focus of the observed teacher's 
attention m relation to student group arrangement. Differences in group structure call 
tor ditterent instructional and group management techniques. Conversely, differences in 
instructional intent may call for different arrangements. Thus, evaluation of one ^ s 
teaching techniques necessitates identification of the group structure 



10 NULL 



No app-rent structure; the teacher doesn't have 

a group of student; classification is not relevant in 

this situation. 

Examples: teacher not in room; teacher is 
observing the cooperating te ache r /class ; 
teacher not responsible for anj^ student. 



11 WHOLE GROUP 
RESPONSIBILITY 



The teacher is directing a task in which all students 
from the class are simultaneously involved. 
Exam ples: lecture; demonstration; class discussion; whole 
group is involved in seatwork with teacher monitoring. 



17 PARTIAL GROUP 
RESPONSIBILITY 



The teacher is working with one group (subset of 

whole class) and has no responsibility for other students 

in the room. 

Example^, teacher is working with one small group on an 
assignment while rest of group is watching a demonstration 
presented by another teacher. 



13 PARTIAL GROUP 
RESPONSIBILITY 
WITH ADDITIONAL 
MONITORING 



The teacher's primary responsibility is inscruction of one 
group (subset of whole group) , but he/she is also 
monitoring the task behavior of other students in the 
classroom. 

Example: teacher is helping one group with their joint 
science project while monitoring other groups who are 
discussing their own projects. 



14 INDIVIDUAL 

RESPONSIBILITY 



The teacher is working with an individual student and has 
no responsibility for the other students m the classroom. 
Exam£le| teacher is tutoring a student while rest of group 
IS participating in a group discussion led by another 
individual . 



15 INDIVIDUAL 

RESPONSIBILITY 
WITH ADDITIONAL 
MONITORING 



The teacher is working with an individual student but is 
also responsible for monitoring the task behavior of 
other students in the classroom. 

Exarap^ej^ teacher is answering a student's question while 
monitoring other students doing individual seatwork. 
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19 CAN'T TELL Class is in session, but the group structure is not 

immediately determinable. 

Example s : activity has just begun and the group structure 
has not been determined yet; observer enters during a 
session already m progress and the nature of the teachers 
responsibilities is not immediately clear , 
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EDUCATIONAL ACTIVITx: Research has shown chac increased classroom time alloca^.ed to 
academic tasks is positively correlated with gains in student achievement. Further, 
active (or direct) instruction by the teacher is more effective than passive (or 
indirect) instruction. Thus, it should be the goal of the teacher to minimize time spent 
on non-acaderaic activities and to increase the amount of active instruction spent in the 
classroom. 



This classification is not appropriate in this situation, 
the teacher doesn't have responsibility for working with 
student(s) at this time (e.g. group structure is null). 
Examples : class is not in session; teacher is 
observing cooperating teacher/class. 



21 ACTIVE The teacher is engaged in direct, interactive instruction 

INSTRUCTION of the student(s) on a curriculum-related topic. 

Exam p les : lecturing; demonstrating,^ questioning; 
answering students * questions . 



22 PASSIVE The students are engaged in an instructional task but tho 

INSTRUCTION leacher is not actively or directly involved in instruction 

and is not in a position to directly evaluate student 
performance . 

Examples : teacher is monitoring individual seatwork and 
only interacting with a student when he/she asks a 
question; teacher sits at her desk and 

occasionally looks around the room while students take a 
test . 



23 TRANSITION/ During transitions, the teacher and students 

PROCEDURAL are moving from one activity to another. This usually 

involves putting away materials for previous task and 
preparing for the nex-... Procedural activities involve 
recordkeeping and routine classroom tasks. 
?M"?Pi4S^ passing English papers to the front and getting 
out Science folders; waiting for teacher to pass out 
papers; setting up equipment for next activity; 
free time; recess; taking attendance. 



Class is in session, but the educational activity is not 
immediately determinable. 

Ejcam^lesj^ the observer lacks sufficient proximity to the 
activity; the nature of the teacher/student interaction 
is unclear. 
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TEACHER INSTRUCTIONAL BEHAVIORS: This 
the observed teacher that are directly 
content . 



classification focuses on those behaviors of 
related to the delivery of the instructional 



The teacher is noc engaged because he/she has no 
responsibility in the classroom. 

Eiiani£j£Sj_ cooperating teacher is in charge of all pupils 



PREPARATION/ Teacher is engaged in an activity necessary to the 

ADMINISTRATIVE delivery of instruction, or is involved in a routine 

^^"^^^ or administrative classroom task. Teacher is not 

interacting with student(s) in a curriculum-related 

task . 

Examples : resource teacher reads student * s ma. ^ assign- 
ment from the regular class prior to assisting him/her in 
^.ompleting the assignment; teacher writes a series of 
examples on che board prior to leading a class discussion 
on the examples; taking attendance; collecting iunch money; 
reading/listening to announcements. 



32 OBSERVATIONAL 
MONITORING 



The teacher is directly observing how well the stadent(s) 
is/are doing on an curriculum-related task but is not 
overtly interacting vith the student(s). 
Examples: looks over student's shoulder as s/he works; 
watches students work problems on board; listens to oral 
book report. 



33 STRUCTURING/ 
DIRECTING 



Teacher structures or ^ives directions for a 
curriculun-re lated task, or teacher gi^;es rationale 
for doing the task, or teacher relates present activity 
to prior one. Does not involve the substance 
of the task itself. ^' 
Example "Do the first 5 problems on page 22 in your 
math book."; "The reason we're doing this activity is so 
you will Know if you receive the correct change when you 
pay for something at the store;" "Remember, yesterday we 
learned how to use the scale at the bottom of the map to 
find distances between cities. Today ^e'U try out what 
you learned on a map of Indiana..." 



34 EXPLANATION/ 
QUESTIONING 
- PLANNED 



Teacher provides a statement o. asks a question concerning 
tho substance of a curriculum -re lated task. The statement 
or question is not about directions to or structure of 
the ^task. 

Examples^ lecturing; modeling; demonstrating; 
reviewing; "What is nhe capital of Indiana?"; 
the main points of the story."; teacher shows 
and waits for a response." 



"Summarize 
flash card 
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STUDENT(S) BEHAVIORS (Target group): This classification is M.«d 

related behavior of those students (whether an individual, partial group or whole 
group; who are the target instructional focus of the observed teacher. If the teacher is 
also responsible for monitoring other students (Group Str ucture categories 13 and 15) 
the next classification, SUidenU^Behavior^ is used to describe the 

casR-related behavior of the monitored students. 



This classification is not relevant at this time as the 
student(s) are not expected to be engaged in an 
ins true tional ac t ivity . 

Class is not in session; free time; teacher 
is engaged in academic preparation while students wait. 



41 ENGAGED 
- ACTIVE 



The student(s) are overtly attending to or involved in a 
classroom task in an observable manner. 

Example s^ writing; manipulating objects; using calculator; 
typing on computer; asking or answering question; drawing; 
commenting; presenting oral report; reading aloud. 



42 ENGAGED 

- PASSIVE 



The student(s) are covertly attending to or involved in a 
classroom task. 

I?S2£]11L silent reading; listening; watching; thinking. 



43 NON-ENGAGED 
- ACTIVE 



The student(s) are not attending to a classroom task and 
are actively interfering with that task 
(either for themselves or others). 

niaking irrelevant comments; out of seat; 
physically disturbing other students; 



44 NON-ENGAGED 
- PASSIVE 



The student(s) are covertly not attenaing to a classroom 
task . 

Examg le s : daydreaming looking out window; sleeping; 
remaining silent or immobile during oral or written 
assi gnments . 



49 CAN'T TELL An instructional activity is in progress but the nature 

of the student's behavior is not immediately determinable 
E^UPPllL observer lacks sufficient proximity to determine 
student behavior; nature of expected student response is 
not clear. 
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35 EXPLANATION/ 
QUESTIONING 
NEED 



Teacher provides a statement concerning the substance, 
or procedures of a curriculum-related task because 
one or more students have asked for assistance or 
guidance. NOTE: Expl ana t ion/q ue s t ioninp - nianned tends 
to occur during active instruction; whereas 
explanation/questioning - need tends to occur during 
passive instruction , 

Exam£lesj_ Teacher helps Jane because she has raised her 
hand during seatwork. Juan coraes up to che teacher *s desk 
and asks for help with his homework, Tony asks the 
teacher to clarity how he is do complete his seatwork 
as signment • 



36 EVALUATIVE 
FEEDBACK 



Teacher informs the student(s) whether her/his performance 
in a curriculum-related task is correct or incorrect but 
provides nc additional feedback. 

Examples : oral remarks such as "That's right.", "No."; 
written marks on a test or worksheet; physical gestures 
such as "Thumbs up," a nod, or shaka of the head. 



37 TASK ENGAGEMENT: 
FEEDBACK 



Teacher comments on level or degree of student engagement 
or nonengagement, but not about the substance or directions 
of a curriculum-related task. 

Exaing2e£2. "I'm glad to see you're working so hard."; 
Pay attention."; "Quiet, get back to work ..." 



38 BEHAVIORAL 
FEEDBACK 



Teacher comments on student behavior not related to 
specific task-engagement or non-engagement. Includes 
both positive and negative feedback. 

Exampjejj^ "Bryan, I like the way you always come to class 
prepared with your supplies."; "please put your gum away. 
You know It's net allowed in this class."; teacher 
lectures student(s) on wny tripping; a student in the 
aisle is dangerous. 



39 CAN'T TELL 



Teacher is engaged in anr instructional activity, the 
nature of which is not immediately determinable.' 
E_xam£les: The instructional activity has not gone on 
long enough to make clear its nature; lihe observer lacks 
sufficient proximity to the activity to determine the 
teacher ' s behavior . 
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STUDENT(S) BEHAVIORS (Monitored group): This classification is relevant only in 
conjunct ^n with Grouo^Struc tore categories 13 and 15, and is used to descr L the 

a't^e^'JLls r ""'7''' \' ^^^'^^^^ '^'^^ -^^^-^^ -^i^^ the her' prima^y 

respon Ibi U " ''''' ^^^^^^^ '''' ^ additional monitoring 

responsibil the category m :his classification will always be NULL. 



50 NULL 



This classification is not relevant at this time as the 
student(s) are not expected to be engaged in an 
instructional activity . 

Example S2 Class is not in session; free time; teacher 
IS engaged in academic preparation while students wait. 



51 ENGAGED 
- ACTIVE 



The student(s) are overtly attending to or involved in a 
classroom task in an observable manner. 

^.^Piiil writing; manipulating objects; using calculator; 
typing on computer; asking or answering question; drawing; 
commenting; presenting oral report; reading aloud. 



52 ENGAGED 

- PASSIVE 



m a 



The student(s) are covertly attending to or involved 
classroom task . 

Examplej2. silent reading; listening; watching; thinking. 



53 NON-ENGAGED 
- ACTIVE 



The student(s) are not attending to a classroom task and 
are actively interfering with that task 
(either for themselves or others). 

li^Ples_^ making irrelevant comments; out of seat; 
physically disturbing other students; 



54 NON-ENGAGED 
- PASSIVE 



The 



student(s) are covertly not attending to a classroom 



task. 

Example£2^ daydreaming; looking out window; sleeping- 
remaining silent or immobile during oral or written 
assignments . 



CAN T TELL An instructional activity is in progress but the nature 

of the student's behavior is not immediately determinable 
ExampJ.esj_ observer lacks sufficient proximity to determine 
student behavior; nature of expected student response is 
not clear. 
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CHECKLIST OF TEACHER BEHAVIORS 



TEACHER BEHAVIOR: INSTRUCTION 



01 Conveyance of importance of curriculum content 

02 Conveyance of importance of pupil performance 

03 Use of transition time 

04 Degree of task-orientation 

05 Organization of time 

06 Organization of presentation 

07 Organization of materials 

08 Sequencing of instruction 

09 Pacing of instruction 

10 Accuracy of information presented 

11 Goal-d irec tedness of instruction 

12 Use of a multicultural approach to instruction and curriculum 

13 Use of a non-discriminatory approach to instruction and curriculum 

14 Clarity of directions 

15 Use of different modes of instruction 

16 Diversity of questioning patterns 

17 Pro/ision of prompting cues to pupils 

18 Ac cnowledgemen t and use of pupil input and ideas 

19 Provis ion of feedback about pupil performance 

20 Provision for basic skill acquisition 

21 Enhancement of pup lis* knowledge base 

22 Provis ion for s uc ce ss f u 1 e xpe r ie nee s 

23 Provision for practice of acquired skills 

24 Provision for mastery of acquired skills 

25 Provis ion for ove r lapp ing of acquire d skills 

26 Provision for transfer of learning of skills 

27 Provision for language and vocabulary development 

28 Provision for pupil decision-making 

29 Awareness aad accommodation of individual differences 

30 Educat iona 1 re leva nee/ soundness of ins true t ion 

31 Relevancy of materials and instruction to curriculum 

32 Comprehensiveness of curriculum 

33 Age-appropriateness of materials and instructional techniques 

34 Ability-level appropriateness of materials and instructional tech.'iques 

35 Interest-appropriateness of materials and instructional techniques 

36 Stimulus value of materials and instructional techniques 

37 Use of varied instructional resources 

38 Appropriateness of oral language patterns 

39 Legibility of writing 

40 Appropriateness of written language 
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TEACHER BEHAVIOR: BEHAVIOR MANAGEMENT 



50 Awareness of classroom dynamics 

51 Clarification of behavioral expectations 

52 Control of behavioral antecedents 

53 Use of reinforcement 

54 Effective use of punishment 

55 Use of non-verbal cues 

56 Consistency in use of behavior management techniques 

57 Adaptability co changing situations 

58 Directiveness and assert iveness 



TEACHER BEHAVIOR: AFFECTIVE /SOCIAL 

70 Sensitivity to pupil needs 

71 Ability to relate to pupils 

72 Communication of concern and support for pupils 

73 Objectivity and fairness in interactions with pupils 

74 Modeling of social and p:rsonal behaviors 

75 Confidence displayed 

76 Enthusiasm for role 

77 Use of humor 

78 Moderation of voice (tone and volume) 

79 Cooperation with other school personnel 

80 Obtainment of needed information and assistance from others 

81 Communication with others about pupils (objectivity, discretion) 

82 Positive interaction with pupils 



STUDENT TASK ATTENTION 

90 Degree of student task-orientation 

91 Affective response of pupils 

92 Level of group participation 

93 Relevancy of participation 

94 Variety of participation modes 

95 Degree of student task success 



STEEL OBSERVATION SYSTEM 



OPERATIONAL GUIDELINES FOR 
CHECKLIST OF TEACHER BEHAVIORS 



TEACHER BEHAVIOR: INSTRUCTION 



01 Conveyance of importance of curriculum content: - Teacher communicates 
to students the importance of instructional activities. Teacher informs 
students of how instructional activities relate to other activities, 
both in and outs Lde school . 



02 Conveyance of importance of pupil performance - Teacher communicates 
to students that the quality of their performance during instructional 
activities is important. Teacher communicates why it is important to 
perform well during instructional activities. 

03 Use of transition time -- Teacher keeps transition time to a minimum. 
Transitions are smooth and effective. Transitions are made at 
appropriate times. Teacher uses transition time to accomplish desirable 
outcomes (e.g establish relationships with students, provide feedback, 
etc . ) . 

0^ Degree of task~orientat ion - Teacher keeps students involved and 
on-task during instructional activities. Teacher achieves lesson goals 
and objectives as planned. 

05 Organization of time - Teacher allocates appropriate amount of time 
for instructional activities. Lesson is implemented in manner that uses 
time effectively. 

06 Organization of presentation - Lesson is presented in coherent, 
logical, and and organized manner. Teacher organizes the classroom 
resources and students, as necessary, to achieve instructional goals. 

07 Organization of materials - Materials used during instructional 
activities are coherent and logical. Teacher has gathered necessary 
materials before beginning lesson. Materials used during instructional 
activities are readily available to students and teacher. 

08 Sequencing of instruction - Lesson is presented in a sequential 
manner. Sequence of instructional activity is appropriate for situation 
and students. Lesson is sequenced in manner that maximizes student 
involvement and attention. 

09 Pacing of instruction - Pace of instructional activity is appropriate 
for situation and students. Pace of lesson maximizes student involvement 
and attention. 
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10 Accuracy of information presented - Information presented to students 
is factually correct. Explanations are clear and accurate. 

16 



11 Goal-directedness of instructiou - Lesson addresses a relevant and 
desirable instructional goal. Activities are directly related to planned 
goals and objectives. 

12 Use of a multicultural approach to instruction and curriculum - 
Materials and instructional presentation makes reference to different 
ethnic groups. Different cultural orientations and values are 
acknowledged during instructional activities. 

13 Use of a non^discr iminatory approach to instruction and curriculum - 
Teacher equitably distributes time, attention, materials, and privileges 
among all students. 

14 Clarity of directions - Teacher provides appropriate and clear 
directions. Students understand directions and expectations. 

15 Use of different modes of instruction ~ Teacher uses different modes 
to present instruction, e.g. seatwork, chalkboard, discussion. Teacher 
appropriately varies mode of instructional presentation throughout 
lesson . 

16 Diversity of questioning patterns - Teacher uses questions that tap 
different modes of thinking, e.g. convergent, divergent, factual, 
inferential, evaluative. Teacher uses questioning patterns appropriately 
for situation and students. 

17 Provision of prompting cues to pupils - Teacher uses cues and prompts 
rather than always providing a direct answer to students. Teacher uses 
cues and prompts appropriately for situation and students. Teacher 
varies cues and prompts (e.g. verbal, physical, gestural). 

18 Acknowledgement and use of pupil input and ideas - Teacher 
communicates to students that their ideas and statements are important. 
Teacher incorporates student ideas in instructional activities. 

19 Provision of feedback about pupil performance - Teacher provides 
immediate and appropriate feedback to students. Feedback is relevant and 
informative. Feedback enables students to correct their mistakes. 
Teacher varies manner in which feedback is provided. 

20 Provision for basic skill acquisition - Teacher plans and implements 
activities that promote the acquisition and practice of basic skills, 
e.g. reading, language arts, mathematics. Activities for basic skill 
acquisition are appropriate for situation and students. 

21 Enhancement of pupils* knowledge base - Teacher plans and implements 
activities that expand pupils* store of general information and 
knowledge. Activities designed to enhance students* knowledge base are 
appropr ia*--- -'or students and situation. 

22 Provision for successful experiences - Lessons and activities are 
leveled and paced so that students may experience success. Teacher 
provides support and assistance as necessary so that students can 
succeed. 
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23 Provision for practice of acquired skills - Teacher plans and 
implements activities that provide additional and repeated practice in 
skills that have been previously caught. Sufficient amounts of practice 
are provided. 

24 Provision for mastery of acquired skills - Teacher plans and 
implements activities that encourage overlearning and mastery of desired 
skills. Teacher evaluates students to determine mastery. 

25 Provision for overlapping of acquired skills - Teacher plans and 
implements activities that require the practice and application of more 
than one acquired skill. 

26 Provision for transfer of learning of skills - Teacher plans and 
implements activities that encourage generalization and transfer of 
skills to new applications and/or settings. Activities designed to 
encourage transfer are appropriate for situation and students. 

27 Provision for language and vocabulary development - Instructional 
activities enhance students* language skills and vocabulary development. 
Teacher encourages students to practice and use appropriate oral and 
written language and vocabulary. 

28 Provision for pupil decision-making - Teacher permits students to 
make their own decisions where appropriate. Teacher provides options or 
alternatives for students- Decision-making is informally or formally 
taught . 

29 Awareness and accommodation of individual differences - Instructional 
activities and materials are tailored to the characteristics of 
individual students - including age, interests, and ability- level . 

30 Educational relevance/soundness of instruction - Teacher utilizes 
appropriate and sound instructional practices (e.g. feedback, success, 
reinforcement). Instructional activities are relevant, 'valuable, and 
worthwhile for students. 

31 Relevancy of materials and instruction to curriculum - Within a 
lesson, materials used and instruction presented are related to 
curricular goals and objectives. Instructional activities provided over 
time form a relevant and coherent curriculum. 

32 Comprehensiveness of curriculum - The overall curriculum is of 
appropriate breadth and depth. Teacher plans and implements 
instructional activities for all appropriate curricular areas. 

33 Age-appropriateness of materials and instructional techniques - 
Materials and instruction are appropriate for students* chronological 
age. 

34 Ability-level appropriateness of materials and instructional 
techniques - Materials and instruction are appropriately matched to 
students* abilities. 



35 Interest-appropriateness of materials and instructional techniques - 
Materials and instruction are appropriately matched to the interests of 
individual students . 

3^ Stimulus value of materials and instructional techniques - Teacher 
uses materials and instructional techniques that capture and hold 
students ' attention. 

3^ Use of va ried instructional resources - Teacher uses varied equipment 
and materials in the presentation of instruction (e.g. printed 
materials, media, computers, guest speakers, field trips, etc.). 

3^ Appropriateness of oral language patterns - Teacher uses appropriate 
and correct oral language, including grammar and dialec^. Teacher sets a 
good example for appropriate oral language usage within the classroom* 

3^ Legibilit y of writing - Teacher's handwriting (in instructional 
presentations, materials, communication, and on the chalkboard) is neat, 
correct, and legible. 

Appropria teness of written language - Teacher uses appropriate and 
correct written language, including style, grammar, and ^ .tence 
structure. Teacher is a good model of appropriate written language usage 
within the classroom. 

TEACHER BEHAVIOR: BEHAVIOR MANAGEMENT 

50 Awareness of classroom dynamics - Teacher demonstrates an awareness 
of students' roles and interactive behavior within the classrooTi. 
Teacher is aware of relationships and interactions among students and 
between him/herself and students. Teacher manipulates classroom dynamics 
to minimize disruptions and to achieve desirable outcomes. 

51 Clarification of behavioral expectations - Teacher clearly and 
directly informs students of expectations for their classroom behavior, 
and of consequences for violating these expectations. 

52 Control o f behavioral antecedents - Teacher sets up the classroom to 
promote appropriate behavior (e.g. physical arrangement, seating 
arrangement, location of materials). Teacher uses resources and 
materials to avoid potential behavior problems. 

53 Use of rei nforcement ~ Teacher appropriately uses praise and rewards 
that reinforce desirable behavior. Teacher does not reinforce 
inappropriate behavior. Teacher varies schedules and types of 
reinforcement . 

5^ Effective use of punishment - Teacher uses punishment in appropriate 
situations. Punishment is applied efficiently and fairly. Punishment is 
applied with a minimum of interruption and disruption to classroom 
activities. The punishment chosen by the teacher suits the situation. 
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55 Use of no n-verbal ci^es - Teacher uses physical, gestural, and 
body-language cues to encourage'' des irable behavior and prevent /stop 
undesirable behavior (e.g. raising eyebrows, pointing, proximity 
control )• Students attend to and understand teacher's non-verbal cues* 
Non verbal cues accomplish desired outcomes • 

56 Consistency in use of behavior management techniques ~ Teacher is 
consistent in his/her expectations, use of reinforcement and punishment, 
and behavioral management strategies. Teacher's behavior roanagement 
techniques are consistent across time and across individuals, 

57 Adaptabi lity to changing situations - Teacher adapts his/her actions 
to changing classroom conditions and individual student needs. Teacher 
is flexible in usa of behavior management strategies, depending on the 
situation at hand, 

58 pirectiveness and assert iveness - Teacher is direct and assertive in 
his/her dealings with students and other school personnel, 

TEACHER BEHAVIOR: AFFECTIVE/SOCIAL 

^0 Sensitivity to pupil needs - Teacher communicates a recognition of 
individual needs and feelings. Teacher responds to individual needs and 
feelings when interacting with students, 

71 Ability to relate to pupils - Teacher has established a rapport with 
students. Teacher is able to relate to pupils from different social and 
cultural backgrounds , 

72 Communication of concern and support for pupils - Teacher's attitudes 
and behavior toward students communicates acceptance, understanding, and 
warmth. Teacher demonstrates that he/she cares about how well student 
perform, how much they learn, how they feel, and how they interact with 
others . 

73 Objectivity and fairness in interactions with pupils - In all 
interactions with students, the teacher remains objective and fair. 
Differential treatment is not applied unfairly to specific individuals. 

7^ Modeling of social and personal behaviors - In his/her interactions 
with others, the teacher sets a good example of appropriate and 
equitable social behavior. 

75 Confidence displayed - Teacher displays a confident demeanor in the 
classroom and school building. Teacher seems secure in his/her role, 

^(> Enthusiasm for role - Teacher appears pleased with his/her role and 
enjoys teaching. Teacher talks favorably about school events, 

77 Use of humor - Teacher uses hum.or appropriately and effectively. 
Humor achieves desirable outcomes, e.g, increases the stimulus value of 
activities, enhances student rapport, 

78 Moderation of voice (tone and volume) - Teacher's voice quality is 
appropriate for the situation and students. Teacher's voice is pleasing, 
clear, understandable, and nondistracting, 
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79 Cooperation with other school personnel - Teacher establishes 
cooperative relationships with other teachers and srhool personnel. 

xeacher works with nth 

j,^o4.wj.vc <juuc<jiiies ior scuaencs ana 

tor the school m general. 

Obtain.jen t of needed information and assistance from o thers - 
Teacher ccnmunicates and cooperates with others to obtain uTf^mation 
and assistance that will ultimately benefit students. Teacher asks for 
information and assistance in appropriate situations. Teacher is 
assertive and direct in requesting information/assistance. 

81 Coimnunication with oth ers about pupils (objectivity, discreti on) - 
Teacher knows about and adheres to legal guidelines regarding 
communication of pupil information. Teacher speaks about students in a 
positive and constructive manner. Teacher shares information as 
appropriate . 

^2 Positive interaction w ith pupils - Teacher-student interactions are 
positive and pleasing. Teacher and students share mutual respect for one 
another. 

STUDENT TASK ATTENTION 

90 Degree of student tssk- or ientat ion - Student on-task behavior is 
mximized. Students apply an appropriate amount of effort to classroom 
activities. Student remain attentive during classroom activities. 

91 Affective response of pupils - Students appear to enjoy classroom 
activities. Students appear to like and respect the teacher. Students 
are eager to participate in class. Student speak favorably of class 
activities and the teacher. 

?2 Level of group participation - All students are attentive and 
involved in group activities. All students have a chance to participate. 
All students behave appropriately. 

Relevancy of participation - Students' participation is relevant to 
the goal? and objectives of the lesson or activity. Students make a 
worthwhile contribution to the activity. 

94 Vari ety of participation modes - Students can respond to 
instructional activities in more than one mode, e.g. oral, written, 
gestural, manipulative. A variety of response modes are used throughout 
the lesson. 



^5 Degree of student task success - All students are able to experience 
some degree of success during a classroom activity. 
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Software Development. A second major accomplishment during the 
first year of the STEEL project involved the writing of software 
packages to allow the collection of trainee behavior using a portable 
microcomputer. The portable microcomputer was necessary in order to 
permit field observations, an^ to provide accurate data storage and 
immediate feedback to teacher trainees on their teaching performance. 
After evaluating the merits of various portable microcomputers available 
at that time, the Epson HX-20 was selected. This is a lap-sized, 
sophisticated, reliable instrument which is easily programmable in 
BASIC. Its features make it an ideal data collection instrument: it has 
an internal clock to signal beginning and ending observation intervals, 
an earphone jack to provide cues to observers, microcasset te audiotape 
data storage, a small dot matrix printer, programmable function keys, 
and a 20-character by four line LCD display window. Software packages 
designed for the Epson were written so that the observation system could 
be altered easily. 

Observer training and field testing. After the software for the 
microcomputer observation system was completed and field tested, a 
second step involved developing a procedure for training field 
observers. First, observers were required to read an observers manual 
and pass a criterion-referenced test covering the various definitions on 
the observation system (see Attachnient A this section). Second, an 
operations manual related to use of the Epson HX-20 was written by the 
project staff and issued to observers (see Attachment B this sectio.;. 
Observers were required to read this manual and become familiar with the 
operations of the microcomputer and its use in observational activities. 
Third, observers underwent laboratory training to learn how to code 
teacher-pupil interactions. This procedure involved having observers 
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STEEL 

Portable Computer Operation Guide 
for the 

Observation Data Collector Program 



by 

Sharon Goh and Bob Eckert 
CITH / Indiana Universifv 



This guide IS intended to serve as a quick-reference for the 
operation of the EPSON HX-20 Portable Computer and its accessories in 
conjunction with the STEEL Observation Data Collector software. This 
guide IS divided into 3 sections which are described below: 

Section I 

Inside and Outside the EPSON HX-20 

on ^hP^I;y%r'J'°".^""'^^! """"^ important 'outside buttons and knobs' 
on the HX-20. how they are 'pushed and turned', and their function. A 
short description of the 'inside' of the HX-20 is included to enlarge 
the observers awareness and understanding of how observational data is 
captured, processed and stored within the machine. 

Section II 

STEEL Observati on Data Collector Program Functions 

This section describes how to activate the STEEL Observation Data 

sforllT f u functions, and manage the 

storage of observation data on microcasset te data tapes. 

Section III 

Care and Feeding of the HX-20 

This section describes the procedures for changing microprinter 
paper microprinter ribbons and charging the HX-20's rechargeable 
nickel-cadmium batteries. 
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Section I 
Inside and Outside the EPSON HX-20 

The EPSON HX-?0 Portable Computer is a totally self-contained 
battery-Dowered 32K memory portable computer, complete with microprinter 
for printed output, a 4 line by 20 column LCD (liquid crystal display) 
viewscreen and microcasset te recorder for data/program storage. A unique 
feature of the HX-20 is its 'non-volatile' memory. This simply means 
that ttie contents of the memory of the HX-20 (which may be holding 
programs and/ or data) is NOT destroyed when the power switch is placed 
in the OFF position. A tiny amount of battery power is used to 
refresh the memory until the unit is used again. This preservation of 
oeiDory is very reliable. The unit can be left with the power off for 
long periods without disturbing the memory contents. 

Below is a summary of the relevant operational controls and 
accessory attachment jacks on the outside of the HX-20: 

f ON/OFF =. This slide switch is located on the right-hand side 

ot the HX-20. When it is pushed towards the rear of the HX-20, power is 
applied to the microprocessor and the LCD display. The HX-20 will beep 
twice and a menu of options will appear on the viewscreen. 

VIEWING ANGLE: This rotary thumbwheel is located on the right-hand 
side of the HX-20 just to the rear of the power switch. It is rotated to 
adjust the viewing angle of the viewscreen in order to provide the 
clearest display for the observer under different lighting conditions. 

PRINTER ON/OFF: This small slide switch is located jus^ below and 
to the left of the microprinter. When this switch is in the 'off 
position, the microprinter is unable to print. Any data which the HX-20 
IS attempting to print will simply be ignored by the printer. When the 

^T- u ,w on^ °"' P^^il^i^"' the printer will be able to print data 

%Aich the HX-20 sends to it . 

PAPER FEED: This small rectangular pushbutton is located iust to 
the right of the PRINTER ON/OFF switch (see above). If the PRINTER 
ON/OFF switch IS in the 'on' position and the PAPER FEED button is 
depressed, the paper in the microprinter will be advanced 1 line at a 
time for the amount of titre the button is held down. This button has no 
effect If the PRINTER ON/CrF switch is in the 'off position. 

_ AC ADAPTOR JACK: This jack is located on the back of the HX-20 and 

".t «v n ^""r^^- '° ^"^'^'^ ^"^^ <=harger unit to 

the HX-20 either for recharging the internal batteries or for using the 
HX-20 on wall power for short periods should the unit ask to be 
recharged at an inconvenient ti.me. 
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uv on ' recessed button is located on the right-hand side of 

Sprim^^ANrp^ T^" ^^^SED UNDER ANY 

CIRCUMSTANCES. Pressing this button causes ALL of the programs or data 
«hich may be m the HX-20's internal memory to be destroyed. The button 
IS recessed to prevent unintentional activation. 



or 
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Section II 

STEEL Observatio n Data Collector Program Functions 

Follow the procedure belcw to activate the STEEL Observation 
Data Collector software: 

Turn on the power switch. When the program menu is displayed on 
the Viewscreen. select option number 3 (labelled COLLECT) by pressing 
the number <3> key. In a moment, the STEEL Observation Data Collector 
^^tlm "^^^ f*Jentify itself on the viewscreen. This indicates that the 
STEEL Observation Data Collector software has been activated. 

_ On the third line of the viewscre. -i will be the name and/, 
version of the observation system which has been placed into the 
internal memory of the HX-20. 

On the fourth line will be a prompt for the observer to select a 
Junction. If you are finished using the HX-20, you may turn the power 
switch off at this time without disturbing any data which you may have 
collected but which has not been recorded on a tape in the microcassette 
recorder. 

Notice the row of keys below and just to the left of the 
viewscreen. They are labelled PFl to PF5 . PF stands for programmed 
function. Just above these keys are alternate labels for these keys. 
These alternate labels are the functions of the STEEL Observation Data 
Collector software. You will notice that there are two functions assigned 
to function keys PFl and PF5 . The top function on each of these keys in 
activated by pressing either <SHIFT> key and simultaneously pressing the 
desired key. These iop functions of each of these keys are thus called 
sbxrted ' functions . 

The pages which follow constitute a summary of the STEEL 
Observation Data Collector function keys. 
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****************** 

The CLEAR Function 

****************** 



This function is used to erase any data which may be in the' 
internal memory of the HX-20 . This function DOES NOT erase any data 
which may be recorded on a tape in the microcassette recorder. The HX-20 
wj.ll confirm any request to clear data before the erasure is actually 
done . 



***************** 

T>-i TAPE Function 

***************** 



"^^^^ ^^^^ Function offers a choice of 2 sub funct ions : SWITCH and 
CATALOG. The SWITCH subfunction is used to insert or remove 
mxcrocassettes into or from the microcassette recorder. The CATALOG 
subfunction IS used to produce a listing of the data files on a given 
tape which is currently in the microcassette recorder. 

Below is a detailed description of each of TAPE ' s subfunrtions : 
The SWITCH Subfunction 

When selected, the SWITCH subfunction first checKS to see if a 
tape IS currently in the microcassette recorder. If none is found, the 
observer is prompted to place a microcassette (either new or old) in the 
recorder. 

The observer is then asked whether the tape which has just been 
inserted is NEW (has never been recorded on, or has been recorded on and 
IS to be reused as if it were new). 

If the observer responds t^-at the tape is new, SWITCH will ask for 
the observer s code number, the tape number and the side (A or B) o'^ the 
tape. This information is called the TAPE HEADER. It is used to identify 
the ^ape which is currently in the microcassette recorder to the STEEL 
Observation Data Collector software. Following this the tape is rewound 
and the TAPE HEADER is recorded .t the very beginning of the tape! 
The TAPE HEADER is also retained in the memory of the HX-20. 

If the obst.ver responds that the tape is not new, SWITCH rewinds 
the tape and reads the TAPE HEADER into the memory of the HX-20. 

The CATALOG Subfunct ion 

The CATALOG subfun^H^ is used to produce a printed listing of 
the data files on a given tape which ic currently in the microcassette 
recorder. When selected, CATALOG asks the user to turn the printer on 
and ready the printer. The user is then asked to indicate completion of 
this task by pressing <Y> for YES on the keyboard. 
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CATALOG then produces a printed copy of the TAPE HEADER from the 
memory of the HX-20. Included on the printout are the observer's code 
number, the tape number, and the side (A or B) of the tape, followed by 
a list of data files previously recorded on the tape. Information 
provide, for each data file consists of the datafile number (1-5); date 
of observation, observer code number, and the code number for the 
teacher or subject observed. 

When printing is completed, the CATALOG asks the user to turn off 
the printer and indicate completion of this task by pressing <Y> for YES 
on the keyboard. 
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The OBSERVE Function 

******************** 
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..Hin/ i 5""<=tion IS used to code real-time observation data. Prior to 
coding, header information is entered consisting of the current date, 
he observer number and the subject number. This information is used to 
Identify the data which is to be coded. Items in the header which have 
already been specified may be skipped by pressing the <RETURN> key. An 
inc'widual Item in the header (such as the date) may be corrected bv 
IZTlli f ^ <CLR> key. retyping the item and pressing the <RETURN>\ey. 

cJass Hcati -"fr^'K^" ^ '""'^ ^^'^^^^ ^or each 

mnF^ Aff observation system. These are called the START 

begins. '"'^^ entered, real-time coding 

HOW CAN'T TE L L CODES ARE HANDLED 
TFTT <=l^^^ifi<=^tion which contains a code corresponding to CAN'T 

^mt % r . ^ '° "^'"'^^"'^ ^"^^ ^^'P'^^y °f the classification 

number followed by a question mark as in: 1? 2? 3? This example 
indicates that classifications 1. 2. snd 3 all coatain a CAN'T TELL 
fwM^h ^ non-CAN'T TELL code for a classification 

dlsappe^::'' '^""''^ possible), this display gradually 

FEATURE CODES 

ronp.Q """^^^ '° ''^''^ reserved for use as FEATURE 

ind/!r'dP..'H T ^^^"^ '""'"'^^ -'^^ t° -^^tivate 

and/or deactivate various features of the STEEL Observation Data 

Collector software while an observation is taking place. These feature 
codes are described below. i^dLure 

FEATURE CODES 01 and 02 

SHOWIHG THE CURRE NT CODES FOR EACH CLASSIFICATION 

K o ^L^l '° '^"'^'^^"'^ ^^^"^ <:ode for each classification 

by 2 methods: ON-DEMAND STATUS or AUTO-STATUS. The selection 

these methods is handled by 2 feature codes. 01 and 02, respectively. 

If the observer wishes to see the current codes for each 
classification on-demand, then a 01 is entered as a code. Since this code 
IS not a valid data code, the program will interpret this code as a feature 
.ode and display the current data codes for each classification 

If the observer wishes to see the current codes for each 
classification automatically after each data code they enter, then a 02 is 

th"Tu?0-ST:T5s'f:ar\°'^''^^^^"''^ ^^"'^^ ^ ^'^e- a 

the AUTO ^>TATUS feature i, now ON. and the current data codes for 

each classification will be displayed. To turn off the AUTO-STATUS 

fs noToFF -ll"' ' "''J ^ -"--g^ th^t the feature 

IS now OFF will appear. Since this code is not a valid data code, the 
program will interpret this code as a feature code. 

Please note that feature codes 03 to 09 are as yet undefined but 
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equent versions of che STEEL Observation Data Collector 



STOPPING CODING 

If an event occurs during coding which necessitates that coding be 
halted temporarily then the observer should enter a 00 code. At this time 
the internal clock of the STEEL Observation Data Collector softwaife is 
stopped and the observer is asked whether coding is to resume. If the 
observer is finished coding, the question should be answered N for NO. If 
the halt was temporary and coding should resume the observer should 
answer Y for YES. 
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The CHECKLIST Function 

This function is u<?ed to code checklist data. Prior to coding, 
header information is entei-d consisting of the current date, the 
observer number and the subject number. This header information is 
identical to the header requested by the OBSERVE function. An item in 
the header may be changed by pressing the <CLR> key, typing in the 
desired value for the item, and then pressing the <RETURN> key. Items in 
the header which have already been specified via the OBSERVE function 
(and which do not require alteration) may be skipped by pressing the 
<RETURN> key as each item is displayed in turn. 

Once the header information has been entered, changed or skipped, 
checklist coding begins. Checklist items may be coded in any order. 
Items which have already had a value specified by the observer will be 
shown with the previous value in place. 

ENTERING A VALUE FOR A CHECKLIST ITEM 

A value for a checklist item should be initially entered by first 
selecting the item, typing in the desired value, and pressing the 
<RETURN> key. 

CHANGING A VALUE FOR A CHECKLIST ITEM 

A value for a checklist item may be changed by first selecting the 
item, pressing the <CLR> key, typing the new value for the item, and 
pressing the <RETURN> key. 

ERASING A VALUE FOR A CHECKLIST ITEM 

A value for a checklist item may be erased entirely by first 
selecting the item, pressing the <CLR> key and then pressing the 
<RETURN> key. 

STOPPING CODING 

When the observer is finished coding, a checklist item number of 
00 is entered. The screen then returns to a display of the master 
"Funccion?*' prompt. 
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The REPORT Function 



This function is used to obtain a printed listing of the 
Observational data stored in the HX-20 ' s internal memory. The REPORT 
function may be used immediately after an observation and before the 
current data has been SAVED onto tape, or later when the datafile of 
interest has been reLOADED into memory from tape. There are three 

ri;pS^'^cT°"\^"''" ""^^ OBSERVATION subfunction, the 

tHtCKLIST subfunction, and the BOTH subfunction. 

The OBSERVATION Subfunction 

selected, the OBSERVATION subfunction produces a printed 
summary of the percentage of time each category in th- five 
classifications was observed during the total observation period. This 
information is preceded by a listing of the DATAFILE HEADER information, 
including date of observation, observer code number, subject number, and 
Tri ll r observation. Only categories actually entered 

during the observation are reported. Printing of the observation report 
may be stopped at any point by typing an <S> for STOP. 

The CHECKLIST Subfunction 

The CHECKLIST subfunction provides a printed listing of the 
checklist Items chosen by the observer, along with the rating given the 
ub ect on each of the chosen items. This information is preceded b/a 
listing of the DATAFILE HEADER information including date of 
observation, observer code number, and subject number. Printing of the 
checklist report may be stopped at any point by typing an <S> for STOp! 

The BOT H Subfunct ion 

includ^'both'^.'^'^K '''' ^-bfunction provides a printed report wich 
includes both the observation summary and the checklist listing. Order 
xn which information is presented is as follows; DATAFILE HEADER 
observation summary, and checklist listing. Printing of the reports may 
be stopped at any point by typing an <S> for STOP. NOTE: Typing <S> 
during the printing of the observation Summary will result in the 
machine skipping ahead to the checklist listing. <S> must be typed again 
if this section of the report is not desired. ^ 
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The SAVE Function 



This function is used to transfer or SAW data which is curi'entlv 
in the internal memory of the HX-20 to a microcassette tape which has 

TlTE/l^Uci wJio::"°"^^"^^ ''''''''' 

When the SAVE function key is pressed, the HX-20 checks to see if 
a tape is in the recorder and if a storage position is available on the 

use^hP ^A^^%'''^'' °' ^^^^^ P^^^^P^^ ^he user to 

use th^TAPE_function_to insert a new tape (or turn it over). If the 
tape IS present and is not full, the HX--0 positions the tape 
automatically at the approppriate location and records the datafile onto 

„,>h TAPE HEADER in the HX-20 ' s internal memory is also updated 

with the record. When the SAVE is finished, the screen returns the 
T Li"""'''""- important to remember that at the end 

of a SAVE operation, the internal memory has been cleared to accept new 
observation data. ^i-i-eFi. new 



■k*-k-k^-k*ic-k*-ki,-M->:-k-k-k 

The LOAD Function 

**-k*-k*-k'k-k-k-k-k-kie^-k-k 



This function is used to transfer or LOAD a datafile from a tape 
in the microcassette recorder to the internal memory of the HX-20. If 
to T^An'^f '"^^T? ^lf^fy'<=ontains data, the user will not be allowed 
to LOAD and will be asked to CLEAR or SAVE the data in the memory before 
attempting to LOAD again. ' 

When the LOAD function key is pressed, the user is asked for the 
number of the file to be loaded. Since each side of a tape holds five 
complete observations, this number will be from 1 to 5. (If the user doPc. 
not know the file number, the TAPE function and its CATALOG 
subfunction will provide a printed listing of the datafiles on the tape). 

1 '"^^ ^^P^ positioned automatically and 

the datafile is loaded rnto memory. NOTE: Only a copy of the datafile is 
loaded into memory. The original datafile is still present on the tape The 
datafile is now ready to be REPORTED, EDITED, and/or reSAVED (if 
changes were made. ) 
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Section III 
Care and Feeding of the EPSON HX~2Q 

WHAT WILL I NEED to Care for and Feed the HX-20? 

You should have some quantity of the following accessory items for 
the HX-20: 

1) A 24-hour timer unit. 

2) An EPSON AC Adaptor/Recharger Unit, 

3) Extra HX-20 Microprinter paper, 

4) Extra HX-20 Microprinter ribbons. 

The following pages discuss setting up, when and how to recharge 
the batteries in the HX-20 ; how to change the microprinter paper; and 
how to change the microprinter ribbon. 
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_S ETTING UP TO RECHARGE Che Batteries in tha HX-20 
m-oa'l^-T 2U"^ouT timer unit for tinning the recharging of the 

HX-20 s internal batteries, perform the following steps: 

1) _ Locate the 24-hour timer unit in your group of HX-20 ' 
accessories It is a small gray box with an AC plug on its back, an AC 
outlet on Its top, and a large round dial on its front. 

2) There are 2 small plastic tabs (each a different color) iutting 
out from the the large round dial on the front of the timer. These tabs 
determine the time at which the timer will switch 'ON' and pro^^iHe power 
to the AC receptacle on its top, and the time at which the ti aer will 
r^ew^M.f s '° '""^ receptacle. An individual tab may be moved to 
a new time by pressing down on the tab gently with your thumb and 
Sliding the tab to tne desired new time around the dial. Please check 
and make certain that the 'ON' tab is set for 7 PM and the 'OFF' tab is 
III 'nl< '^"'"^ ^ charging time of 8 hours. You may set 
the ON' and 'OFF' times to different (possibly more convenient) 

'ON''and''^^;^^°" ""'^ '"'^^'^ difference between the 

ON and 'OFF' times is EXACTLY 8 HOURS. 

VERY IMPORTANT! 
THE RECHARGIN G PERIOD MUST NOT BE LONGER THAN 8 HOURS: 

3) Once the 'ON' and 'OFF' tabs have been adjusted, turn the large 
round dial CLOCKWISE until the CURRENT TIME is lined up ^itb the raised 
plastic arrow on the right next to the dial. Then plug the timer i^ o a 
convenient electrical outlet, preferably higher than floor level, ill 
timer will now turn on and off at the 'ON' and 'OFF' times which have 
been set, whether or not the AC Adaptor/Recharger unit is plugged into 
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WHEN TO RECHARGE the Batteries in the HX-20 

The HX-20 will notify you that its batteries should be re-charged 
soon by interrupting whatever you are doing and flashing the message 

CHARGE BATTERY!' on the viewscreen for approximately 30 seconds. 
Then the viewscreen will return to displaying whatever was on it before 
the interruption took place. If this message is displayed, you should do 
the following: 

1) Turn off the power switch of the HX-20. 

2) Plug in Lhe AC Adaptor /Recharger into an electrical ouclet. 

3) Plug the other end of the AC Ad£p;:or /Recharger into the jack on 
the back of the HX-20 labelled: AC ADAPTOR. 

4) Turn on the HX-20, re-run the STEEL Observation Data 
Collector program (see SECTION II), and resume your work. It does not 
harm the HX-20 to be powered by its AC Adaptor /Recharger unit for 
short periods. However, you should NOT use the AC Adaptor /Recharger 
until the HX-20 has told you to 'CHARGE BATTERY!'. 

IMPORTANT 

If you are using the OBSERVE function when the 'CHARGE BA-^TEKY'' 
message appears (extremely unlikely), you will NOT be able to resume 
coding where you left off. You will, however, be able to use all of th- 
other functions, once the AC Adaptor/Recharger has been connected. 

5) Later, at your borne, at the end of the day, you should recharge 
the batteries of the HX-20 using the AC Adaptor/Recharger and the 
24-hour timer unit. 
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STEEL Portable Computer Operation Guide 
Observation Data Collector Program 



HOW TO RECHARGE the Batteries in the HX-20 

IMPORTANT 

MAKE CERTAIN THAT THE POWER SWITCH OF THE HX-20 IS IN 
THE 'OFF' POSITION BEFORE YOU RECHARGE THE BATTERIES. ' 

To recharge the batteries of the HX-20 at home, perform the 
following steps: 

1) If you have NOT setup the 24-hour timer unit at home to perform 
the recharging process, please refer to the section above, titled- SETTING 
UP TO RECHARGE THE BATTERIES IN THE HX-20. Then proceed with 

the next step below. 

2) Plug the AC Adaptor/Recharger unit into the 24-hour timer at or 
before the ON time for which the timer has been set. Then plug the 
other end of the AC Adaptor/Recharger into the jack on the back of the 
HX-20 labelled: AC ADAPTOR. At the 'ON' time, the timer will switch on, 
and the AC Adaptor/Recharger will begin recharging the batteries in the 
HX-20. At the 'OFF' time (which MUST be 8 hours later), the timer will 
shut off. This enables you to perform the recharging process without 
having to worry about whether or not EXACTLY 8 hours have elapsed. The 
timer will take care of this for yuu. 

3) After the timer has shut off, you may disconnect the AC 
Adaptor/Recharger from the timer and from the HX-20. The HX-20 's 
batteries are now fully recharged and ready to go! 

PLEASE NOTE 

Please remember that whatever data and/or programs may be in the 
internal memory of the HX-20 when the 'CHARGE BATTERY" message is 
displayed, IS PROTECTED. You will most likely be interrupted with the 
message when more battery power is being called for by some section of 
tne HX-20. Two sections of the HX-20 are likely to trigger the 
interruption: the microprinter and the microcassette recorder. Both of 
these devices contain electric motors vhich require much more battery 
power to operate than the rest of the computer. 
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CHANGING the PAPER .n d RIBBON in ^hP. HX-20 Microp rinter 
Please refer to the small blue and white booklet titled: 
Operations Manual/EPSON HX-20 Notebook Computer, and read pages 15 
(starting at 'Preparing the Printer for Use') to 23 (ending at 
Adjusting the Viewing Angle') for information on changing the ' 
microprinter paper and ribbon. 
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simultaneously view videotapes of classroom interactions and to enter 
teacher and child behavior codes via a numeric keypad into a 
microcomputer. The microcomputer instantaneously compared each observers 
input and halted tne progression of the videotape when coders failed to 
agree. This procedure required observers to discuss discrepancies in 
their codes to reach a consensus on the proper codes for a particular 
behavioral sequence. This process continued until all coders reached 90% 
agreement in at least one training session. A criterion observer on the 
CITH staff also served periodically as a field observer. Observer drift 
was prevented by having the criterion observer simultaneously code with 
each individual observer in actual field settings. If interobserver 
agreement dropped below 80^ in the field with the criterion observer, an 
observer was recalibrated with additional laboratory training. 

In the course of development of the observation sy.tem, procedures 
for conducting observation sessions were also developed. After the 
graduate supervisors were given several training sessions in how to use 
the Epson HX-20's to code observation data, they took the machines into 
the public school classrooms. As practicum experiences were the most 
appropriate settings in which to evaluate the observation system, the 
graduate-student supervisors conducted field tests in classroom 
settings. Observing teacher trainees while they taught, the graduate 
supervisors entered into the machine the appropriate codes from the 
observation system. Using data taken from the observation system, the 
supervisors provided feedback to teacher trainees regarding their 
classroom performance in bi-weekly individual training sessions. 

Trainee Feedback. Preservice teacher trainees received regular 
feedback regarding their effectiveness in providing ALT related 
instruction in the field. The observational summary printouts generated 
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by the computer served as the focus for these sessions. In the 
observation sessions, the supervisor or practiciim instructor identified 
the teacher trainee's rppropriate instructional and classroom management 
behaviors, and when necessary, the behaviors which .ney needed to 
change. For example, after several observation periods a supervisor 
might indicate in a feedback session that the student trainee used 
adequate rates of positive feedback, but that she or he needed to 
increase rates of explanation/questioning-need and decrease 
preparation/administrative duties in the classroom. This procedure, 
which was followed throughout the semester, provided a continual source 
of constructive fe.^dback. It not only identified the specific behavior, 
students needed to change, it allowed student trainees to set personal 
goals for maintaining or increasing appropriate teaching behaviors and 
for reducing levels of inappropriate teaching behaviors. 

Not only can data from student behavior be recorded, individual 
data is automatically stored on microcassette tapes resident in the 
computer itself. Information on these tapes then may be aggregated 
across student trainee records to yield a composite of group data. These 
data then may be used for evaluating the performance of an entire group 
of students and represents one aspect of the program evaluation function 
the STEEL/ IMS offers. 

To illustrate how the data may be used, the observational data 
obtained during four observation periods during the first year of the 
Project on the initial group of teacher trainees are presented in Table 
1. Teacher educational activities consisted primarily of "active 
instruction" (74.7%) and "passive instruction" (15.82). The active 
instructional time was divided between "explanation-planned" (41.9%), 
"observational monitoring" (16.7%), "structuring/directing" (11.6%), and 
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tabu: 1: 1983-84 LARGE GROUP DATA 



VARIABLE 

Group Structure Null 

Group Structure Whole Group 

Group Structure Partial Group 

Group Structure Partial Group Additional Monitoring 
Group Structure Individual Responsibility 

Group Structure Individual Responsibility Additional Monitoring 
Group Structure Can't Tell 
Educational Activity Null 
Educational Activity Active Instruction 
Educational Activity Passive Instruction 
Educational Activity Transition/Procedural 
Educational Activity Can't Tell 
Teacher Instructional Behaviors Null 
Teacher Instructional Preparation Supervision 
Teacher Instructional Behaviors Observational Monitoring 
Teacher Instructional Behaviors Struct uring/ Direct ing 
Teacher Instructional Behaviors Explanation/Planned 

Behaviors ExplanaLion/Need 
Evaluative Feedback 
Task Feedback 
Behavioral Feedback 
Can't fell 



Teacher 
Teacher 
Teacher 
Teacher 
Teacher 
Primary 
Primary 
Primary 
Primary 
Primary 
Primary 
Secondary 
Secondary 
Secondary 
Secondary 
Secondary 
Secondary 



Instructional 
Ins true tional 
Ins true tional 
Ins true tional 
Instructional 
Student Null 
Student 
Student 
Student 
Student 
Student 
Student 
Student 
Student 
Student 
Student 
S tud^nt 



Engaged-Active 
Engaged-Pass ive 
Non Engaged-Active 
Non Engaged-Passive 
Cant't Tell 
Null 

Engaged-Active 
Engaged-Pass ive 
Non Engaged-Act i ve 
Non Engaged-Pass ive 



Can't Tell 



N 


* 

WbAN 


* 

STANDARD 
ntr T A T T r\Kt 


1 90 
1 Z ^ 


Z . U 


1 U . 0 


1 9Q 


Do * H 


A A 1 




1 A A 


J J • 1 


1 '^O 

1^7 


^ 7 


zU . U 


1 90 

1 / 7 


A A 
4 . D 


1 O . J 


1 90 
i Z7 




x3 .9 


1 9 0 

1 Z 7 


U . J 




1 90 
1 Z7 


Z . 1 


7 . 4 


1 9 0 

1 Z 7 


7 A "7 


oft 1 

29 . 1 


1 Z 7 


1 J . o 


25 . 7 


1 9 0 

1 Z 7 


a Q 
0 . 7 


13.3 


1 90 


U . D 


3 .0 


1 9 0 
1 Z 7 


1 . 4 


6 . 3 


1 9 0 
1 Z 7 


/ o 
^ . J 


8. 9 


1 90 

1 Z 7 


1 <^ 7 

lo . / 


20 . 6 


1 9 0 
IZ 7 


i 1 . D 


14 . 4 


1 90 

1 Z 7 


A 1 O 
41 • 7 


29 . 4 


1 9 0 

1 Z 7 


1 n 7 


1 Q A 

1 o . 4 


1 9 0 

1 Z 7 


A 0 


9 . 6 


1 90 
1 Z 7 


J - U 


o . 9 


1 90 

1 Z 7 


A 0 

H . 7 


9 . 2 


1 9 Q 

1 Z 7 


U . 0 


3 . 2 


1 90 
1 Z 7 


z . J 


7 . 4 


1 90 

1 Z 7 


JO . 1 


35 . 6 


1 90 

1 Z 7 


00 A 
ZV . H 


34 . D 


I Z 7 


7 9 
/ . Z 


1 D . Z 


1 90 

1 Z 7 


A ') 
H« Z 


1 U . Z 


1 90 

1 Z 7 


n 0 

U . 7 


4 . D 


129 


88.9 


27.0 


129 


6.6 


20.7 


129 


1.5 


9.2 


129 


0.4 


2.6 


129 


1.8 


9.7 


129 


0.8 


4.6 



Table 1 (cont.) 



VARIABLE 



ITEMl 


/rrv 
U7 


ITEM 2 


7/. 
IH 


ITEM 3 


0 / 


ITEM4 




ITEMS 


on 


ITEM6 


0 J 


ITEM 7 


79 


ITEMS 




ITEM 9 


7A 


ITEMIO 




ITEMll 


fin 


ITEMl 2 


/ip 


ITEMl 3 


9P 

zo 


ITEM14 


Q7 


ITEM15 


67 


ITEM16 


H- J 


ITEMl 7 




ITEM 18 


7^ 


ITEMl 9 


O J 


ITEM20 


'^Q 


ITEM21 


JL 


ITEM22 


77 


ITEM23 


SI 


ITEM24 




ITEM25 




ITEM26 




ITEM27 




ITEM28 


56 


ITEM29 


DO 


ITEM30 




ITEM 31 


9 


ITEM 32 


0 


ITEM33 


07 


ITEM 34 


oo 


ITEM 35 




ITEM 3 6 


79 


ITEM37 




ITEM38 


A 1 


ITEM39 




ITEM40 


D 

c 


ITEM41 


n 
u 


ITEM42 


0 


ITEM43 


0 


ITEM44 


0 


ITEM45 


0 


ITEM46 


0 


ITEM47 


0 


ITEM48 


0 


ITEM49 


0 


ITEM50 


96 


ITEM51 


105 


ITEMS 2 


73 



MEAN STANDARD 
DEVIATION 
2.3 0.7 

2.5 0.6 

2.7 0.6 

2.8 0.5 

2.6 0.6 

2.6 0.6 

2.8 0.5 

2.7 0.6 

2.6 0.6 

2.7 0.7 

2.7 0.5 

2.9 0.4 

2.8 0.4 

2.5 0.6 

2.6 0.6 
2.5 0.7 

2.7 0.7 
2.7 0.5 

2.7 0.5 

2.9 0.2 

2.8 0.4 

2.8 0.4 

2.9 0.3 
2.9 0.3 
2.8 0.4 
2.8 0.4 
2.8 0.4 
2.8 0.4 

2.8 0.5 

2.9 0.4 
3.0 0.0 

2.5 0.5 
2.9 0.4 

2.8 0.5 

2.9 0.3 

2.7 0.6 

2.8 0.5 

2.6 0.6 

2.9 0.3 
2.6 0.7 



2.5 0.6 

2.4 0.7 

2.5 0.6 
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Table 1 (cont) 



VARIABLE N MEAN STANDARD 

DEVIATION 



ITEM53 


98 


2.3 


0.7 


ITEM54 


29 


1.7 


0.8 


ITEM55 


66 


2-5 


0.7 


ITEM36 


82 


2.4 


0.7 


ITEM57 


56 


2.6 


0.6 




104 


2.7 


0.5 




0 


• 




TTPMAH 


U 


• 


• 


TTPM A ] 


U 


• 


• 




0 


• 


• 


TTPM A 
JLILrlO J 


0 


• 


• 


TTPM AA 


0 


• 


• 


TTPM 


0 


• 


• 


TTPMA A 


0 


• 


• 


lliLWD / 


0 


• 


• 


TTPM Aft 


0 


• 


• 
• 


TTPM 


0 


• 


• 


iicn / U 


o/. 


2.7 


0.5 


TTPM7 ] 


1 A Q 

lOo 


2.9 


0.3 


TTI7M7 0 


■JO 

79 


2.8 


0.4 


TTPM?'^ 
JLX Crl / J 


Q 1 

ol 


2.9 


0.3 


ilEM/4 


81 


2.9 


0.4 




99 


2.8 


0.4 


ilcM/D 


97 


2.8 


0.4 


ilfcM/ / 


47 


2.4 


0.7 


ilEM/o 


91 


2.6 


0.6 


I1EM79 


39 


2.9 


0.4 




Q 
O 


3.0 


0.0 




3 


2 . 7 


0.6 




IU5 


2 .8 


0.4 




20 


1 .4 


0.7 




A 
U 


• 




ITEM85 


n 


• 


• 
• 


ITEM86 


0 


• 




ITEM87 


0 


• 


• 
• 


ITEM88 


0 


• 


• 


ITEM89 


0 


• 


• 


ITEM90 


98 


2.5 


0.6 


ITEM91 


90 


2.5 


0.6 


ITEM92 


80 


2.5 


0.6 


ITEM93 


55 


2.6 


0.6 


ITEM94 


59 


2.6 


0.7 


ITEM95 


80 


2.7 


0.5 



A dot (.) signifies that because an item was never coded, it is not possible 
to compute a mean or standard deviation. 

See Appendix A for descriptions of coding categories and checklist items. 
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"explanation-need" (10,7%), while "evaluative feedback" and "behavioral 
feedback" were each provided 4,9^ of the time during this period. 
Although ideal rates of teacher behavior in these categories based upon 
children's achievement gains have yet to be determined, available data 
indicate that, in general, higher rates of active instruction are 
positively related to student achievement gains. Thus, students falling 
below group norms in the future might be counseled to concentrate their 
efforts on increasing specific aspects of their active instructional 
behaviors. Following the actual recording of teacher behavior, 
supervisors also rated student trainees on a 95-item checklist of 
general teaching skills (see Attachment A), Table 1 shows the average 
group ratings obtained on the checklist items for the first year of the 
project , 

Year 2 Activities 
* ■ — ■ 

STEEL/ MBOS Revisio n, In the second year of the STEEL project, a 
revision of the STEEL/MBOS was undertaken. With the feedback received 
from the observers and the practicum instructor at the end of the 
previous year, the observation system was re-evaluated and the necessary 
changes were made. These modifications made the system more responsive 
to the needs of the field-based observers as well as provided more 
complete feedback to the teacher trainees. 

Observations were begun early in the first semester of classes at 
Indiana University, Prior to field observations, four training sessions 
were held with six student teaching supervisors. These sessions included 
general orientation to and discussion of the STEEL project goals and the 
STEEL observation system; discussion and illustration of STEEL 
classifications, categories, and checklist; and instruction in the use 
of the Epson HX-20 portable microcomputer. After these initial 
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discussions, student teaching supervisors were provided with ample 
opportunities to practice using the observation system and the Epson 
HX-20 by coding videotaped segments of teacher-student classroom 
interactions. A more complete description of specific training 
activities is provided in Attachment C, entitled Training Activities 
Schedu le . 

On the fourth day of training, a 30-rainute videotape was coded for 
the purpose of establishing reliability among student teaching 
supervisors in the use of the STEEL observation system. Reliability was 
estimated by using the Flanders reliability coefficient formula 
(Flanders, 1967). This analysis yielded an overall reliability 
coefficient of .58. A closer examination of the data indicated that 
coding of student behavior as engaged active vs. engaged passive and 
engaged vs. non-engaged were major sources of disagreement. Therefore, 
it was decided that additional training of observers was necessary to 
resolve these discrepancies. 

A fifth training session was then held, during which categories of 
student behavior and their operational definitions were reviewed and 
discussed. Following this discussion, a second reliability check was 
taken and a Flanders coefficient of .88 was obtained. Since Flanders 
coefficients above .75 are generally considered adequate (Frick & 
Semmel, 1978), training was conside.-ed complete after this fifth 
session. The six supervisors and practicura instructor then began 
classroom observations of seventy-eight undergraduate trainees enrolled 
in field experience and student teacher coursework. As the STEEL 
observation system is involved so closely with the undergraduate special 
education teacher training program, a description of the field 
experience component will be included in this report. 
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Appendix C. Training Activities Schedule for STEEL Observer Training. 

Training Session #l~August 29, 1984 — 1-4 p.m. 
a* Introduce STEEL Project goals & objectives, 
b* Introduce STEEL observation system. 

c. Discuss STEEL classifications and categories. 

d. Discuss STEEL checklist. 

e. Discuss checklist items and develop operational definitions for 
checklist items. 

Training Session #2~August 31, 1984—9-12 p.m. 

a. Distribute checklist operational definitions and personnel code lists. 

b. Review categories and classifications. 

c. View videotapes and practice coding with observational system and 
checklis t . 

d. Demonstrate operational procedures for EPSON HX-20. 

e* Practice coding, reporting, and saving data on EPSON HX-20. 

Training Session #3— September 5, 1984--8 : 45-10 : 45 a.m. 

a. Distribute revised STEEL categories and operational definitions and discuss 

b. Review operation of EPSON HX-20: discuss problems and questions. 

c. Practice coding with videotapes. 

Training Session #4 — September 10, 1984—8-9 a.m. 

a. Review Tape Function on EPSON HX-20. 

b. Practice coding with videotape. 

c. 30 minute reliability check coding videotapes. 

Training Session #5 — October 5, 1984--8-10 a.m. 

a. Discuss any concerns/prob lems regarding use of EPSON HX-20. 

b. Discuss use of STEEL observational categories for student behavior. 

c. 30 minute reliability check coding videotapes. 
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ST EEL Observati--. System Field Tests . The STEEL observation 
system, during tlie years of the project, was an integral part of the 
undergraduate training program in the Indiana University Department of 
Special Education. The feasibility of its use was thoroughly evaluated 
in project years 2 and 3. In the spccnd year of the project, preservice 
teachers were supervised approximately once every two weeks, three to 
four times in each placement site. The faculty supervisor saw them once 
in each placement and other visits are made by graduate supervisors. 
Supervisors scheduled observations in advance and make -chedules 
available to the supervising teachers and students. Supervisors are 
non-participant observers who typically observe students for 15-30 
minutes. Following the observation period the supervisors conferred with 
the teacher and then with the student for approximately 5-10 minutes 
respecti^^ely. More supervision time was allowed for student teachers: 
supervisory sessions usually last about 45-60 minutes for field 
experience students and about 60-90 minutes for student teachers for 
both the observation and conference. 

The STEEL observation system was used during each of these 
supervisory sessions. Observers (supervisors) typically coded relevant 
ongoing behavior immediately upon arriving and enter the checklist codes 
to reflect accurately the classroom climate(s). Supervisors also wrote 
field notes or anecdotal records of occurrences while present in the 
classroom. After the initial behaviors were coded, further codes were 
entered only when changes in behavior categories were observed. This 
allowed time for observers to intermittently make field notes of 
specific behaviors to share with the trainees following the observation 
period. Then supervisors completed the checklist on the computer, and 
left the room to print out STEEL dita. These results were subsequently 

ERIC 5fj 



viii 



shared with both student and supervising teacher. Data was then saved on 
the cassette tape* Cassettes were periodically returned to the STEEL 
research staff who aggregated and stoied data for future retrieval. Data 
summaries for each were made available to MHP faculty. Examples of the 
various ways student data could be summarized appear in the STEEL/IMS 
Volume of this report. 

Applications. The STEEL system, developed with faculty input and 
thus directly related to the goals and objectives of the undergraduate 
program, added a new dimension to field experience supervision. 
Behavioral observation provided a real-time coding of teacher and 
student behaviors. This information was instantly analyzed and 
summarized; the results and data were shared with students, who can then 
see the exact percentage (and amount) of time that they were engaged in 
various teaching activities. Pupil behaviors were precisely determined 
and recorded. Such feedbacl. helped teacher trainees attain high levels 
of engaged time among their pupils (see Year 3 report, this section). 

The checklist of coding categories and teacher behaviors provides 
entry-level students with a detailed and comprehensive set of teaching 
competencies expected of them. Program objectives would thus be 
specifically defined, and familiarity with the categories encouraged. 
Ultimately, upon completion of the program, student progress could be 
examined and evaluated in relation to classroom demands. The behavior 
codes and checklist results provided faculty with information about 
specific and general student teaching proficiencies. 

The STEEL observation system information also served as an aid to 
the faculty coordinator in examining deficiencies and inadequacies in a 
student's program. For example, it revealed a student who may have 
finished his/her junior year without having had an opportunity to 
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provide large group instruction. Such information allowed the 
coordinator to make sure the student is placed in a setting in his or 
her senior year where group instruction was possible. Or again, a 
student who showed certain weaknesses might be placed in a situation 
where she/he could practice relevant skills. The system thus pinpoints a 
student's weak areas and identifies possible remedies. 

In another application, data aggregated over a student's third and 
fourth years can be shared with students for counseling purposes. If a 
student showed strengths in tutor ing-type activities, for example, and 
weaknesses in large group presentation areas, the student might be 
counseled to seek a resource room setting to be more successful in 
his/her first job. 

Graduate Supervisors^. Although the graduate supervisors initially 
approached using the computers with some trepidation, after a few hours 
of successfully working with the STEEL system their apprehension was 
reduced. Reliability training generally took 10-15 hours in a small 
group setting. All supervisors received this training have gained a 
sense of confidence in using the computers which has undoubtedly 
transferred to other situations in which they will be required to use 
computers. This growth in competence in the use of technology was part 
of the stated aims of the STEEL proposal and one of the accomplishments 
that was perhaps most easy to document. Although many graduate students 
have experienced some frustration and occasional problems (e.g., 
computer batteries occasionally needing to be charged in the middle of 
an observation), in general the supervisors found the computers reliable 
and tairly easy to use. During field testing, many changes were made in 
the software to curb problems they had initially experienced. 



ERIC 



Students' Reactions t o STEEL/MBOS . As with the graduate 
supervisors, the undergraduate students were slightly alarmed when they 
heard that a computerized observation system was tcf be implemented 
during supervision. Gradually, they became accustomed to their use and 
were eager to get the information printout the STEEL/MBOS provides. 
Reactions to the STEEL/MBOS data are varied, as evident in the 
evaluation data gathered during the final year of the project. Seme 
students preferred the field note information and others appeared to be 
more interested in the behavioral observations and checklist categories. 
The majority of students indicated that a combination of the two 
feedback forms was optimal in increasing their tea-.hing skiUs. Since 
both methods complement each other, most students took seriou.s interest 
in both forms of feedback. 

Pupils' and Supervi sing Teachers' Reactions to STEEL/MBOS . 
Children and adults in the field placements had various reactions to the 
computers. Comments ranged from "Isn't that cute" to more sophisticated 
questions regarding the computer's capabilities. Most teachers seemed to 
be sincerely interested in the use of the computers and seemed to 
appreciate the information their field students received. 
Year 3 Activities 

Preservice teacher group data. In the thi. 'ear of the STEEL 
project, activities related to the STEEL/MBOS were confined to summative 
evaluation of the system. As with Years 1 and 2 , data were summarized 
regarding the entire preservice teacher group in the department of 
sp-cial education. This summary appears in Table 2 . 

Effectiveness study. in addition to evaluating the performance of 
preservice teachers as a group during the third year of the project, a 
controlled experimental study was conducted by project staff which 
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TABLE 2: Group Performance Data, Year 3 
STESL/IHS Student Observation Data Summary fcr all students 



Period: 


AUG 01, 1985 


to JUL Olt 








"^uuder of Observations (in suiEifnary)t 


5 55 




Amount cf Observation Time (in sumoiary): 


1303'^. 5 minute(n) 


A y O • NuffhPP of n c* a r ;^ t" 1 nn c n*» r C t* H « r> >• • 


7 




Avcitt A fBOunt of 0 b r\/a ^' 1 n n Tinf^ n*»r 's^-nHianf 


: 173.8 (TiinuteCs) 


033ERiAfI0N CODE SL'^^ARY 






GROUP STPUCTURE 




MINUTES 


NULL 


2. ml X 


( 346.4) 


WHOLE GROUP RESPONSIBILITY 


58 .8% 


( 7658.1) 


PARTIAL GROUP FESP CNS I8ILI TY 


17.9% 


( 2336.1) 


PART GRP RESP3NS WITH AODTL MONITORINf; 


6.5% 


< 8*4.4) 


INOIVIDLAL RESPO.^S I3IL ITY 


10 .3X 


( 1339.1) 


INQIV FF:SPCNS WITH AODTL MONITORING 


3.7% 


( 485.8) 


CAN'T TELL 


.2% 


( 24.6) 


EDUCATIONAL ACTIVITY 


PERCENT 


MINUTES 


.MULL 


3.0% 


( 388.9) 


ACTIVE INSTRUCT lO^i 


79.6% 


(10378.2) 


PASSIVE INSTRUCTION 


10 .5^: 


( 1364.5) 


TRAMSITI ON 


h •Oh 


( 887. 8) 


CAN'T TELL 


.1% 


( 15.1) 


TEACHER INSTRUCTIONAL BEHAVIORS 


PERCENT 


MINUTES 


NULL 


2 .5% 


( 325.7) 


PREPARATION-*-SUPERVISinM 


5.1 X 


( 665.4) 


OBSERVATIONAL MONITORING 


23.9% 


( 3112.1) 


STRLCTURING+ DIRECTING 


18.8% 


< 2449.6) 


EXPLANATION*- QUFST I ON IN G- Pi ANMro 


3*3 . o % 


( 5028.9) 


E X P L A N A T 1 0 N ♦ Q U E S r I 0 N I G - N E E G 


2.9% 


( 372.1) 


EVALUATIVE FEED'UCK 


4 


( 579.6) 


TASK ENGAGEMENT FEEDHACK 


1 .7% 


( 218.3) 


BEHAVIORAL FEEDBACK 


2.1% 


( 276.1) 


CAN'T TELL 


.1% 


( 6.6) 


STU0ENT(3) BEHAVIORS (TARGET G??0UP) 


P'^RCENT 


MINUTES 


NULL 


2 .9% 


( 381.7) 


ENGAGED- ACTI VE 


69.0% 


( 3998.4) 


ENGAGEC-PASSIVE 


17.7% 


( 2309.4) 


NON-ENGAGED- ACT17E 


5.0% 


( 651.0) 


NON-EN GAGED- PASSIVE 


5.3% 


( 691.2) 


CAN'T TELL 


.0% 


( 2.9) 


STUIENT(S) BEHAVIORS (MONITORED GROUP) 


PE? CENT 


MINUTES 


ENGAGED-ACTIVE 


69.5% 


{ 961.3) 


ENGAGED-PASSIVE 


17.6% 


( 243.0) 


NON-ENGAGED- ACTIVE 


6-3% 


( 87,5) 


NO N- EN GAGED- PASSIVE 


5,9% 


( 81.5) 


CAN'T TELL 


.8% 


( 10.6) 
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TABLE 2: (cont.) 



STEEL/IfS Student Observation Ch'jckllst Sufrnary for all students 



Period: 

'^lunoer of Students (in summary): 
'^u«bdr of Observations (In sunmary): 
Aiouot of Observation Time (in suaimary> 
Avq. NufTber of Observations per Student 
Aw9« Amount of Observation Time 



AU- 01, 

75 

1303^o5 
7 

per Student: nz,H 



1985 to JUL 01» 1986 



m1nute( s ) 



08se:^vation checklist summary 



TEACHER behavior: INSTRUCTION 

CONVEYANCE OF IMPORTAfJCE OF CURRICULUM 

CONVEY IMPORTANCE OF PUPIL PERFORMANCE^ 

USE OF TRANSITION TIME 

DEGREE OF TA SK- ORI ENT AT I ON 

ORGANIZATION OF TIME 

ORGANIZATION OF PRESENTATION 

ORGANIZATION OF MATERIALS 

SEQUENCING OF INSTRUCTIOfy 

PACING OF INSTRUCT UN 

ACCLRACY OF INPORMATION vESENTED 

G0AL-OIRECTE0NE3S OF INSTRUCTION 

USE OF NULTfCULT APPROACH TO IN3T*CURR 

USE OF NCN-CI3CRIM APPROACH TO IN3T*CURR 

CLARITY OF OIRECriQNS 

USE OF OIFFE^ENT MOOES INSTRUCTION 
DIVERSITY OF QUESTIONING PATTERNS 
PROVISION OF P.-50'lPTiMG CUES TO PUPILG 
ACKNGWLEDGEMENT+JSE OF PUPIL INPUT+IOIAS 
PROVIDE FEEDBACK ON PUPIL PERFORMANCE-' 
PROVISION FOR BASIC SKILL ACOUISITICrj 
ENHANCEMENT OF PUPILS » KHQ'ALZOGZ 8AS^ 
PROVISION FOR SUCCESSFUL EXPERIENCES 

PRACTICE OF ACQUIRED SKILL 
MASTERY OF ACQUIRED SKILLS 
OVERLAP OF ACQUIRED SKILLS 
TRAriSFER OF LRNG OF SKILLS 
LANGUAGE* V0CA8 DE VELOP-'ENT 
PU°I L OECISIO^-MAKING 

OF INDIVIO DIFf^E^ 
OF INSTRUCTION 



PROVIS ION 
PROVISION 
PROVIS ION 
PROVISION 
PRO*'IS ION 
PRO vIS ION 



FOR 
FOR 
FOR 
FOR 
FOR 
FO R 

A<iARENESS+ACC0M0OA TION 

:ductl elev-^sounoness 
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comprehensiveness OF CURRICULUM 
AGE-APPRGPRIAT MAT E R I A L3 + I N ST T'CH.NIQUES 
A8IL-LEVEL APPROP CF MATER TAL3+I NST ^ECH 
INTEREST-AFPROP of MATERIALS4-INST TfCH 
STIMULUS VALUE 0<^ M ATE R I ALS* I N ST ftCH 
USE OF CLASSROOM RESOURCES 



APPROPRIA.ENESS OF ORAL 
LEGIBILITY OF •iR^rI^G 
APPROPRIATENESS OF WRITTEN 



language pattern 
Language 
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TEACHER behavior: BEHAVIOR MANA'^EMENT 
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TABLE 2: (cont.) 

EL/IMS Student Observation CheckMst Summary for alt stud'?nts 



CLARIFICATION OF BEHAVIORAL EXPECTATIONS 
CONTROL CF BEHAVIORAL EXPECTATIONS 
USE OF REINFORCEMENT 
EFFECTIVE USE OF PUMISH'IENT 
USE OF NON-VERBAL CUES 

CONSISTENCY IN USE OF BEH MANAGE TECHS 
ADAPTABILITY TO CHANGING SITUATIONS 
0IRECTIVENES3 ANO ASSERT I VENESS 

TEACHER BEHAVIOR: AFFECTIVE/SOCIAL 
SENSITIVITY TO PUPIL NEEDS 
ABILITY TO RELATE TO PUPILS 
COMMUNICATES CONCERV+SUPPO F T FCR PUPILS 
08JECTIVE+FAIR IN INTERACT WITH PUPILS 
HOOELING OF SOCIAL % PERSONAL BEHAVIORS 
CONFIDENCE DISPLAYED 
ENTHUSIASM FOR ROLE 
USE OF HUMOR 

HODERATICN OF VO I CE (T ONE + VOLUNIT > 
COOPERATION WITH OTHER SCHOOL PERSONNEL 
OBTAINS NEEDED INF0*AS3TST FROf OTH'RS 
TALKS WITH OTHERS REZPUPILS OBJ+OISCRET 
POSITIVE INTERACTION WITH PUPILS 

STUDENT TASK ATTENTION 

DEGFEE OF STUDENT TASK-ORIENTATION 
AFFECTIVE RESPONSE OF PUPILS 
LEVEL OF GROUP PARTICPATION 
RELEVANCY OF PARTICIPATION 
VARIETY OF PARTICIPATION MODES 
DEGREE TF STUDENT TASK SUCCESS 
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evaluated the effectiveness of the STEEL/MBOS. This study, which has 
been sununarized in the overview section, assessed the effects of the 
STEEL/MBOS in producing change in preservice teachers* instructional 
behavior anc' their effects on handicapped pupils' classroom behavior, 
is anticipated that the results of this study will be published in an 
appropriate journal in the field. The following documen. provides the 
rationale, method, and results of this evaluation. 
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ABSTRACT 

Microconputer- Baspd a«5 55 pc cmorv+ r^f T:>,^a<-a-^-,Ti nn^„„u^_- 
Behavior: Objective Evaluation of Skill Development 

Preservice teachers typically receive field note 
feedback regarding their classroom teaching performance. 
In an attempt to standardize feedback, isolate specific 
teacher behaviors for improvement, and collect normative 
data across students that could be used for formative as 
well as program evaluation, this study presented twenty- 
seven special education students objective performance 
information regarding research-based teaching strategies. 
After a baseline phase of six weeks, students were 
randomly assigned to two groups; differential training 
was provided each group. Group A students received 
information regarding effective teaching practices 
involving the concept of Academic Learning Time (ALT) and 
the relationship between this information and a 
microcomputer-based observation system. Subsequent field 
observations for this group provided feedback on ALT 
teaching behaviors, a three-point rating of their 
academic and behavior management strategies, and general 
information from traditional field notes. Alternate 
training received by Group B students involved humanistic 
classroom management techniques; they received only field 
note feedback. Following this first intervention phase. 
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Group B students received the research-based training, 
and a second intervention phase ensued where both groups 
received computer feedback* Thus, a multiple-baseline 
research design was used to examine the effects of the 
specific training/feedback procedures. Data was analyzed 
both between and within groups to determine the effects 
of the differential training. 

The observation system involved a real-time coding 
feature which recorded actual dur'.tion of observed 
behaviors in five different categories. Using this 
procedure, three teacher behaviors and classroom student 
engagement rates were examined. Results demonstrated 
that the computer-based feedback influenced students to 
significantly increase their rates of effective teaching 
behavior, while the field note feedback improved 
performance on ALT behavior categories only slightly. 
Pupil engagement rates also increased, but failed to 
reach the predetermined level of significance. Student 
attitudes toward the computer-based observation system 
were more positive following the ALT training session. 
The results demonstrate that an objective observation/ 
feedback system is highly effective in training 
pre service teachers to utilize specific behaviors in 
thei,? instructional reperto ires , 
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REVIEW OF THE LITERATURE 
Classroom Instruction Research 
Research on Presage Variables 

The history of research on classroom instruction 
is comparatively brief, spanning only about the last 
80 years. Initial efforts in this field concentrated 
on identifying presage variables which affected student 
achievement. Early educational researchers attempted to 
link teacher traits such as intelligence, flexibility, 
emotional stability, knowledge of subject matter, and 
style of teaching to instructional outcomes. However, 
this line of research failed to establish clear 
relationships between teacher characteristics and student 
achievement. Studies in this area generally concluded 
that teacher traits, besides having low stability 
coefficients, did not always correspond x/ith the quality 
of classroom instruction (Gage, 1963; Getzels & Jackson, 
1963; Biddle & Ellena, 1964). 

Other research efforts attempted to discover 
relationships between student characteristics and 
achievement. Socioeconomic status, number of siblings, 
paternal occupation, and IQ were but a few of the 
variables studied. While correlational data were 
obtained for each, none accounted for the majority of the 
variance observed in student achievement scores. The 
general conclusion emerging from this line of research 
was that traits or characteristics, singly or in various 
combinations, could not be used as a valid predictor of 
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teaching effectiveness (Howsam, 1960). 

In the early 1960's. r»l nmm<a-h i r^cr +00+- -uu„ 
launching of Sputnik, and the civil rights movement, all 
influenced the federal government to more closely examine 
the educational process, including teacher e..ucation 
(Schalock, 1983). A number of curriculum reform efforts 
were initiated during this period (Howsam, Corrigan, 
Denemark, & Nash, 1976), but they failed to produce any 
significant or lasting results. Disillusionment with 
educational practices intensified in the mid-1960's, 
when initial reports of Head Start programs indicated 
that the massive funding of preschool programs was not 
paying educational dividends (Good, 1983); a 
controversial report (Coleman, Campbell, Hobson, 
McPartland, Mood, Weinfeld, & York, 1966) cast further 
doubts as to whether variations in teachers' experience 
made any difference in students' academic achievement. 
Resf^ajich on Teacher-Pupi 1 I nteractions 

During this same period, other researchers began to 
investigate the educational process from a different 
perspective. Their research focused on the time students 
spent engaged in mastering academic skills as well as 
their academic progress, judged by ^heir performance on 
standardized achievement tests. Much of this research 
involved observing teacher-pupil interactions: 
objectively evaluating what teachers were doing as they 
went about their teaching and, in turn, what students 
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were doing in response to teacher actions. 

Early contributors to teacher-pupil interaction 
research included Carroll (1963) and Bloom (1964; 1973; 
1977). Carroll (1963) devised a model of school learning 
which posited a link between time and achievement. In 
this model, time was viewed as a critical variable in 
classroom learning and students differed most in the 
amount of time they required to master a given 
instructional objective. Carroll's model is shown 
in Figure 1 . 



Insert Figure 1 about here 



Carroll's formula, simply stated, illustrates that 
the degree of learning equals the time actually spent in 
learning divided by the amount of time needed to learn. 
The parts of the equation are, in turn, influenced by 
other factors, specifically: the amount of time actually 
spent learning depends upon (a) perseverance--the amount of 
time the learner devotes to mastering a task, and (b) 
opportunity--the time and number of trials the teacher 
allows for learning. The amount of time needed for a 
student to learn depends upon (a) aptitude--not the 
student's learning capacity, but the time it takes him or 
her to learn a particular fact or concept; (b) ability— 
the student's mode or style of learning relative to the 
task at hand; and (c) quality of instruction--the 
effectiveness of the teachers' instruction. 
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Amount of time actually 
spent learning 

a ) perseverance 

b) opportunity 



Amount of time needed 
to learn 

a) aptitude 

b) ability 

c) quality of instruction 



Degree of Learning = 



Figure 1. Carroll 



s (1963) Model of Learning 



ERIC 




7 



Perhaps the major contribution of Carroll 
model of learning to present views of the educational 
process involves the reconceptualizat ion of student 
aptitude as a function of time. According to this view, 
time, not innate ability, assumes a critical role in 
achievement. Subsequent research on time in instruction 
and student learning has built upon Carroll's early work. 
For example. Bloom's (1973) analysis of standardized test 
norms at the elementary school level revealed that the 
achievement level attained by the highest 20% of the 
students on a givpu standardized test was attained by 
about 50% of the students a year later, and by about 80% 
of the students two years later. 

The work of Carroll (1963) and Bloom (1973) focused 
research efforts on interaction^ among classroom student, 
teacher, and instructional variables rather than on 
characteristics of students and teachers ( Cruickshank , 
1976; Rosenshine & Berliner, 1978; Brophy, 1979). This 
line of research is often referred to as "process- 
product", where process includes both teacher and st.udent 
behaviors that have mediating effects on stvdent 
performance and the product, or outcome, is represented 
by observe^' changes in student behavior--generally 
increments in perf ormance on s tandardized achievement 
tests. In the earliest stages of this research thrust, 
naturalistic classroom observations were conducted and 
teacher behaviors identified which correlated positively 
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with student performance gains. In subsequent studies, 
these variables were manipulated experimentally to 
determine their effect on student achievement (Stallings, 
1976; McDonald & Elias, 1976; Good & Grouws, 1979). 
These research efforts had two distinguishing 
characteristics: first, they were based upon the 
findings of correlational studies, rather than on 
educational or psychological theory; second, they 
measured the degree to which teacher training 
influenced their actual classroom behavior (Gage & 
Giaconia, 1981 ) . 

One of the most well known studies in this research 
genre was the Missouri Mathematics Project (Good & 
Grouws, 1979). In this study, 40 third- and fourth- 
grade teachers were initially selected for the stability 
of their instructional procedures. The sample was 
divided into two equivalent groups; half received a 
five-page manual containing a system of sequential, 
instructional steps for teaching mathematics, e.g., use 
of demonstrations, provision of uninterrupted successful 
practice, and maintainance of momentum in the classroom. 
The teachers read the manual, received two 90-minute 
training sessions, and proceeded to implement the key 
instructional behaviors in their mathematics lessons. In 
a deliberate attempt to create a Hawthorne effect, 
members of the control group were informed regarding the 
purpose of the study but they did not receive the 
procedures manual and were told to continue to instruct 
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in their own style. Results showed that teachers in the 

treatment groiin 

_ — ^ **»oi,iijr n.ey j.iit> Vxi uc Lionai 

behaviors and that their students' math test scores 

increased significantly, from the 26th to the 57th 

percentile on national norms. These gains continued for 

at least some time following the treatment. Moreover, 

student attitudes were significantly higher among the 

experimental group, and teachers in the project indicated 

that they felt the program was practical and that they 

planned to continue using it in the future (Good, 1983). 

This study was particularly noteworthy because of the 

significant student gain obtained in only four months, 

and because training time and expense was minimal. 

Research on Academic Learning Time 

Another line of educational research which has 

emerged in the last decade involves identifying effective 

teacher behaviors which affect student achievement. One 

of the more widely accepted studies in this area was the 

Beginning Teacher Evaluation Study, or BTES. (Fisher, 

Berliner, Fiiby, Marliave, Cahen, Dishaw, & Moore, 1978). 

After examining the effects of various teacher behaviors 

on student achievement scores in second and fifth grade 

classrooms, these researchers developed a concept +-hey 

termed Academic Learning Time (ALT) . Academic Learning 

Time refers generally to the time students spend actively 

engaged in completing academic tasks with a high rate of 

success (Berliner, 1977). Since this concept was first 
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proposed, there has been virtually universal agreement 
among researchers that ALT is a major influence on 
student achievement (Lomax & Cooley, 1979). 

The first two components of ALT are allocated time, 
the amount of time the teacher sets aside for learning, 
and engaged time, the amount of time the student actually 
spends actively learning. The third component is success 
rate. Each of these aspects of ALT will be discussed. 

Allocated time . Studies have found great variation 
in time allocated for lear; ng in educational settings 
and compared these with student outcome variables. 
Harnischfeger and Wiley (1978), for example, found that 
the length of a school day in the s-\me school district 
varied by 4r m: ■ ^es for two second-grade classrooms. 
First-fe-rade classrooms have been found to vary as much as 
1 hour and 30 minutes in length of school day; secondary 
class periods for remedial reading from 40-55 minutes 
(Stallings, 1975; Stallings, Needels, &Stayrook, 197S). 
Even larger uifferences in time allocated for instruction 
were observed between individual classrooms. The 
BTES (Fisher et al., 1978) results revealed that, in 
fifth grade reading and reading-related instruction, the 
average amount of allocated time varied between classes 
from about 60 minutes to about 140 minutes per day. The 
latter class thus received 48 more hours o*" reading 
instruction over the course of a school year than the 
former. As time spent -n reading lias been found to 
correlate highly with gains in student achievement 
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(Rosenshine & Berliner, 1978; Fisher et al . , 1978), even 
such seemingly minor variations in ir.struct ional time can 
mean critical differences in student achievement. An 
example of the importance of time allocated to academics 
is fodnd in a recent report by Walberg (1985). After 
summarizing 25 research studies, it was observed that 
95.4% of them found time in learning to be positively 
correlated with student gain. 

Engaged time. Allocated time appears to be a 
necessary but insufficient condition for laarning. 
Although time spent on academic subjects has been proven 
an important variable for predicting student achievement, 
studies which isolated and examined only the effects of 
allocated time have failed to obtain significant 
educational results (Rosenshine, 1980). A more important 
factor appears to be student engagement, the amount of 
time students spend actively engaged in a task. This 
variable has consistently been more closely related to 
academic gain than allocated time, yet observed student 
engagement rates have been found to vary widely. Powell 
and Dishaw (1978), for example, reported that the engaged 
time of second-grade students in daily reading lessons 
ranged from 39 to '98 minutes, and that of fifth-grade 
students from 49 to 105 minutes. The BTES researchers 
(Fisher et al., 1978) observed classes that had average 
engagement rates of about 50 per cent during reading and 
math instruction, i.e., students in these classes were 
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attending to their work half of the time. In other 
classes, the average engagement rate approached 90 per 
cent. These results indicate thar, although teachsrs in 
two classes might both allocate the same amount of time 
to reading, one could have almost twice as much student 
engagement as the other. 

As previously mentioned, the variable of student 
engagement has been significantly related to academic 
achievement. Evertson (1982) reported that low-achieving 
junior high students were engaged an average of 40 per 
cent of the time in academic activities compared with 85 
per cent engagement for high-achieving students. 

The consistent findings of these studies indicates 
that it is critical for teachers to strive for high student 
engagement rates. Teachers can and do influence classroorr, 
levels of student engagement. For instance, teachers who 
frequently engage in substantive interactions (e.g., 
explanations, demonstrations, taskoriented questions), 
who emphasize whole group involvement, and who provide 
feedback are most successful in maintaining high student 
engagement during seatwork (Good Sl Grouws , 1979 ; 
Rosens .e , 1980 ) . 

Success rat.e. The third aspect of ALT is success 
rate. Tasks in which students are engaged must be at an 
appropriate level of difficulty. If tasks are too 
difficult, student engagement rates have been found to 
decrease; if -hey are too easy, students do not learn new 
material and engagemeiit rates may decrease. But when the 
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task is at '..he proper level of difficulty, students will 
spend more time on task (Fisher et al . , 1978). 

From the preceding discussion, it oan be seen that 
studies have successfully identified a number of teacher 
behaviors which significantly increase student academic 
achievement (Emmer, Evertson, & Anderson, 1980; Brophy, 
1981). More effective teachers, those whose students 
show greatest academic gains, provide clear instructions 
regarding classroom assignments, actively monitor their 
students as they work, and provide assistance v.'hen it is 
needed. They also spend less time in transitional 
activities, student behavior management, preparation of 
materials, and explanation of assignments. These teacher 
behaviors have all been found to be positively associated 
with student gain (Stallings, Cory, Fairweather, & 
Needels, 1977; Stallings et al . , 1979; Stallings & 
Kaskowitz, 1974 ) . 

Summary of Classroom Instruction Rese a rch 

The studies levieved above indicate that, although 
early educational research was relatively inconclusive 
regarding variables which affected student performance, 
in the past ten to fifteen years researchers have 
accumulated a sm??ll but consistent body of experimental 
evidence demonstrating that certain teacher behaviors 
produce greater academic gains in students, regardless of 
their ability (cf.. Fisher et al . 1978: Stallings & 
Kaskov .cz, 1974; Stallings et al . , 1977; G ge & 
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Coladarci, 1980). Selected teacher behaviors which have 
been demonstrated to have a positive relationship with 
student academic achievement are summarized in Table 1. 



Insert Table 1 about here 



Specific findings in this area show, for example, 
that student engagement rates are related to achievement; 
low student achievement is associated with time spent 
off' task; the amount of teacher time spent on 
instruction-related interactions is positively related to 
student achievement; and the amount of teacher time spent 
on management-related interactions is negatively related 
to student achievement (Wang & Walberg, 1983). More 
effective teachers tend to use active instructional 
strategies such as^ providing clear explanations and 
questions, monitoring student seatwork, providing 
immediate corrective feedback, and reducing the amount of 
transition time in the classroom. These are just a few 
of the conditions necessary for effective teaching and 
learning . 

Teacher E ducation R esearch 

While the research literature has clearly 
demonstrated the potency of certain teaching strategies 
and procedures , preservice teachers rarely receive 
systematic information regarding research-based practices 
(Prehm, 1976, Drew, Preator, & Buchanan, 1982; Lewis & 
Blackhurst, 1983). Rather, techniques and practices are 
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Table 1 

Summary of Studies Which Have Identif ied Effecti 
Teaching Behaviors 

Study Referent 





i 


2 


3 


4 


5_ 




Teacher Questions 


X 


X 


X 




X 




Active Instruction 


X 


X 


X 


X 


X 




Academic Orientation 


X 


X 


X 


X 


X 




More Time Allocated 
for Instruction 


X 


X 


X 


X 


X 




Reduced Transition 
Time 






X 


X 


X 




Student Engagement 


X 


X 


X 


^- 


X 




Positive or Neutral 
Corrective Feedback 


X 


X 




X 


X 




Effective Classroom 
Management Skills 


X 


X 


X 


X 


X 




Teacher Supervision 


X 




X 


X 







Note. l=Stallings & Kaskowitz (1974) 



2-Stallings, Cory, Fairweather, & Need s (1977) 
3=Brophy & Evertson (1976) 
4=Good & Grouws (1979) 

5=Anderson, Evertson, & Brophy (1978) 

( table con t inues ) 

O n J' 



Table 1 



Summary of Studies Which Have 


Identified Effective 


Teaching Behaviors 




Study Referent 






6 


18 9 




Teacher Questions 




X X 




Active Instruction 


X 


X 




Academic Orientation 


X 


XXX 


X 


More Time Allocated 
for Instruction 


X 


X X 


X 


Reduced Transition 
Time 


X 


X X 


X 


Student Engagement 


X 


XXX 




Positive or Neutral 
Corrective Feedback 


X 


X X 




Effective Classroom 

ilclliclgeiUcn L OKIXXS 


X 


XXX 


X 


Teacher Supervision 


X 


X X 




Note, 6=Fisher, et 


al. (1973) 




7=Soar; Soar 


& Soar 


( 1973, 1976) 




8=Stallings, 


Needels 


, & Stayrook { 1979) 





9=Anderson & Evertson (1978) 
10=Gage & Coladarci (1980) 
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typically based upon subjective hunches and hypotheses 
growing out of experience (Cyphert, 1972; Griffin, 1983), 
a situation which has changed little over the years 
(Sykes, 1983). This is a disturbing commentary on the 
status of teacher education, since a large-scale study of 
student t<-aching (Griffin, Barnes, Hughes, O'Neal, 
Defino, Edwards, & Hukill, 1983) and experimental studies 
of inservice teacher education (Griffin, 1983; Wang & 
Walberg, 1983; Gage & Coladarci, 1980; Leach & Dolan, 
1985; Sparks, 1986) all suggest that teachers can become 
proficient users of research-based practices, given 
appropriate information . 

In light of these findings, it seems obvious that, 
to be maximally effective, teacher training programs 
should be based upon the most current research regarding 
effective teaching and learning and should provide 
prospective teachers with clear and consistent feedback 
regarding their performance in applying effective 
teaching behaviors. If increasing student achievement is 
an important educational goal, the tendency for training 
programs to be lax in conveying relevant research-based 
information to students (Lanier & Little, 1986; 
Feistritzer, 1984) seems ill-advised, since research on 
effective teaching has shown that what teachers do in the 
classroom is second only to what students know from 
previous experience in predicting student outcomes 
(McDonald, 1976 ) . 

Thus, teacher behavior is clearly the most pronising 
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manipulable variable discovered to date for improving 
student performance (Peterson, 1986). But to what extent 
have teacher training programs attempted to incorporate 
research results into their training activities? Three 
recent studies bear on this issue. Waimon (1983) divided 
preservice teachers into two groups and provided training 
to only the experimental group. Following training, the 
preservice teachers worked with high school pupils for 50 
minutes, five times per week for three weeks. Student 
academic learning time (ALT) was recorded daily by 
trained obser-ers as well as by supervising teachers. 
The dependent measure was pupil performance on a multiple 
choice test. Students in the experimental group nearly 
doubled their amount of active teaching behavior. While 
scores on the pupil achievement measure favored the 
experimental group, they did not reach a significant 
level. This finding may be due to the relatively short 
duration of the experiment, or to the minor instructional 
role played by the student teachers in this study. 

Madike (1980) assigned stuaent teachers to 
teach five-week mathematics units to comparable ninth 
grade classes. One group of student teachers had 
experienced a microteaching program which trained them in 
specific teaching skills. A second group had been observed 
and given feedback by supervising teachers, but not 
necessarily on the skills stressed in the microteaching 
program. A third group was given no specific preparation 
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for the teaching experience. Each student teacher was 
videotaped during a 35-minute lesson, and a 10-minute 
segment was rated for frequency of use of nine skills 
taught in the microteaching program- Results inaicated 
that the microteaching group had higher frequencies of 
behaviors related to the skills, and that their use 
correlated positively with student achievement. Thus, in 
this study, feedback on instructional performance in a 
laboratory training situation showed positive transfer to 
the actual instructional setting. 

Ponzio (1984) reported a study in which five 
preservice and five inservice teachers were questioned 
regarding their level of awareness of the effective 
teaching research. The cooperating teachers were 
unfamiliar with the research while the student teachers 
were ''relatively unaware" of effective instruction 
practices. After receiving training, changes in 
teachers' use of effective instructional behaviors 
were examined with three different types of instruments: 
pre-post paragraphs written by the participants regarding 
research knowledge and application; pre-post classroom 
observations on teacher behaviors; and teacher 
interviews. Observational data indicated that both 
groups increased their use of effective instructional 
behaviors. Additionally, the cooperating teachers 
unanimously identified ALT as being most useful for 
helping them become more effective in their teaching and 
supervisory roles, while the student teachers unanimously 
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Knowledge and Skill Transfer Research 

Another line of research regarding effective 
teacher education involves variables related tc 
transferring knowledge to actual practice situations. 
After reviewing the training literature, Joyce and 
Showers (1982) reported that four components of 
training— presentation, demonstration, practice, and 
feedback— are sufficient to induce many teachers to 
transfer recommended practices to the classroom, while 
coaching may be a necessary fifth component for some. 
Training teachers to discriminate, generate, and evaluate 
their interactive teaching skills additionally depends 
upon (a) specification of target behaviors, (b) reliable, 
valid performance information, (c) immediate availability 
of feedback information to the trainee, and (d) access to 
data from previous training trials. 

Utilizing this methodological framework, preservice 
teachers increased their use of effective teaching 
behaviors over baseline rates by a ratio of nearly three 
to one (Semmel, Sitko, Semmel, Frick, & Hasselbring, 
1976; Semmel, 1978). in Pre ect CARTLO (Computer- 
Assisted Research into Teaching-Learning Outcomes), 
preservice teachers rated (a) the use of observation data 
to evaluate effectiveness of teaching methods, and (b) 
the use of computers to list ou observation information 
as valuable contributors to their professional growth 
(Rieth & Frick, 197?). The effect of these specific 
teaching behaviors on classroom student engagement rates 
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was not examined. 

Problem Statement 
Now that research findings have accumulated into 
bodies of fairly well-confirmed knowledge (Rosenshine & 
Stevens, 1986; Berliner, 1986), it seems crucial to move 
toward some "goodness of fit" model, wh ire preservice 
teachers are trained to use more effective ^nstn-ct ional 
behaviors. The pivotal question involves how to increase 
preservice and inservice teachers' levels of appropriate 
instruction and corresponding student engagement. The 
answer to this question would avoid what Feiman-Nemser 
and Buchmann (1983) call the "two-worlds pitfall," where 
the world cf thought--university courses--is far removed 
from the world of acti on— school classrooms. Including 
research-based training and evaluation activities into 
teacher education programs could bridge the gap between 
these two worlds. At the same time, it would end the 
lament of many scholars (e.g., Lortie, 1975; Howsam et 
al., 1976) who have noted that teachers have no share-' 
body of technical knowledge upon which to draw for 
structuring their instructional program. 

Assuming that changes in teaching behavior are 
critical to the improvement of teaching, which ultimately 
will impinge upon student achievement, one finds 
suprisingly little systematic inquiry into the process of 
teacher training (Sparks, 1986). Few studies in the 
teacher education literature have been conducted on 
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methods for training preservice teachers to use research- 
based instructional behavio^-s. No studies were found 
which examined the effect of these research-based 
behaviors on student engagement rates. It is critical 
that prospective teachers not only are made aware of 
effective teaching techniques, but that they be allowed 
to develop their ability to apply these behaviors in 
classroom settings and to objectively examine their 
effect on student engagement. This study was therefore 
undertaken to investigate variables which affect teacher 
trainees' acquisition of effective teaching skills. 
Specifically, it addressed the following questions: 

If preservice teachers are taught effective teaching 
skills, expected to use them when teaching, and provided 
feedback on the use of these skills, (a) will they 
demonstrate increased use of these skills in actual 
teaching situations, and (b) do changes in use of these 
skills increase student behaviors associated with higher 
achievement levels? 

Hypotheses 

The specific hypotheses investigated in this study 

were : 

1. There will be no significant difference between 
Groups A and B in observed levels of various teacher 
behaviors during the baseline phase. 

2. There will be no significant difference between 
Groups A and B in observed levels of student 
engagement during the baseline phase. 
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Information regarding effective teaching practices 
and computer-generated feedback will result in 
statistically significant differences betw^ £?n the 
groups in favor of Group A for active teacher 
instruction during the first intervention phase. 
Information regarding effective teaching practices 
and computer-generated feedback will result in 
Stat: i.cally significant differences between the 
groupo in favor of Group A for student engagement 
levels duri-.g the first intervention phase. 
There will be no significant difference between 
Group A and Group B in observed levels of active 
instructi ^ and student engagement during the second 
intervention phase--Group B means on these variables 
will match those of Group A during thi-^ period. 
Group A will show a statistically significant 
increase over rheir baseline level in the categories 
of active instruction and student engagement 
following the first intervention phase. 
There will be a significant increase in Group B's 
acv /e teacher instruction and student engagement 
nean from baseline to the second intervention 
phase . 

Student ratings on the computer checklist items 
will improve as a result of training. 

There will be a statistically significant difference 
between the groups in favor of Group A regarding 
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attitudes toward the microcomputer-based 
observation system following the first training 

session. Di f f <:ir*c -^r^^r^o ^u- _ , 

.WW., ^y^x^fy^^n ^^iit; groups wijLi De 

nonsignificant after the second training session. 
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these students. 

This study was conducted during the fall and spring 
semesters, during which time the students participated in 
three different field experiences. In the first 
semester, there v,ere two different six-week placements in 
the Monroe County Community School Corporation. Students 
were placed in a variety of settings, including preschool 
through high school self-fjontained classrooms, or 
multicategorical resource rooms with Mildly Mentally 
Handicapped, Emotionally Handicapped, Moderately Mentally 
Handicapped, Learning Disabled, and Multiply Handicapped 
children. During the second semester, students 
travelled to the Indianapolis Public School System for an 
eight-week practicum in order to experience a more urban 
school environment. In each of these placements, 
students were expected to assist the teacher with regular 
classroom duties, some of which included: preparing 
lessons, teaching individual students, small groups of 
students or the entire class, grading papers, monitoring 
individual seat work, or assisting with recess and 
special events. 

Students were supervised at least three times in 
each placement, for a minimum of nine observations per 
student. All observations were conducted by graduate 
students in special education. The observers scheduled 
all observations in odvanca and ...ade the schedules 
available to the supervising; teachers and students. 
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Thus, students knew when they would be observed, and 
tried to arrange with their supervising teacher to be 
Conducting a Toe 

_ I. lid u uxiilt;. 

Assignment to Condition 

The participants in this study were randomly 
assigned to two groups. Since differential training was 
to be provided to each group, the students were divided 
on the basis of a pre-existing condition: enrollment in 
two sections of a reading methods course. The students 
had been assigned to the different sections on a random 
basis. Thus, groups were equitably divided according to 
ability. The students were never informed of their 
status in the study, and the observers also were blind to 
condition . 
Observers 

Several graduate students were recruited to observe 
the undergraduates. Criteria for selection of these 
individuals included prior teaching experience as well as 
a personal interview with the field-experience 
instructor. Observers generally worked 10 to 20 hours 
per week and were responsible for supervising 10 to 15 
students each semester. During the course of the study, 
three individuals conducted the student observations. 
Observation ostein 

The Special Teacher Education and Evaluation 
Laboratory (STEEL) observation system was used in this 
study. The observation system is comprised of five major 
categories: Group Structure, Educational Activity, 
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Teacher Ins trucrional Behaviors, Student(s) Behaviors 
(Target Group), and Student(s) Behavior (Monitored 
Group). Each major category contained five or more 
subcategories for coding purposes, all of which were 
operationally defined. The category of group structure 
described the focus of the observed teacher's attention 
in relation to student group arrangement. Subcategories 
included (a) whole group, (b) partial group, (c) 
individual responsibility, (d) partial group with 
additional monitoring, and (e) individual respons ibi 1 ty 
with additional monitoring. This category recognized 
that differences in group structure call for different 
instructional and group management techniques. 
Conversely, differences in instructional intent may call 
for different grouping arrangements. Thus, evaluation of 
teaching techniques nec:3sitated identifying the 
classroom structure. 

The second major observational category described 
the educational activity occurring in the classroom. 
Subcategories included both active and passive 
instruction and transition/procedural. The third 
category was teacher instructional behavior, which 
focused on those activities of the observed teacher which 
were directly related to the delivery of instructional 
content. Tl.ere were eight teacher instructional 
behaviors , including (a) preparation/administrative 
duties, (b) observarionai monitoring, (c) structuring/ 
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directing, (d) explanation/questicning-planned , 
(e) explanation/questioning-need, (f) evaluative 
feedback, (g) rask engagement feedback, and 
(h) behavioral feedback. 

The final two categories of the observation system 
measured classroom student behavior in both the target 
and (if .-applicable) monitored groups. These 
classifications were used to describe the task-related 
behavior of rhose students, whether an individual, 
partial group, or whole group, who were the target 
instructionax focus of the observed teacher. If the 
teacher was responsible for monitoring other students, 
the student behavior (monitored group) category was used. 
Within these categories, student behavior could be coded 
either (a) engaged-active, (b) engaged-passive, (c) non- 
engaged-active , or (d) non-engaged-passive . 

Finally, within each of the five observational 
categories, the observers were provided with two 
additional choices. Null was coded if a given 
classification was not relevant to the observed 
situacion, e.g., if the teacher trainee was not 
responsible for any portion of the class; Can't Tell 
indicated that the observer was not able to immediately 
determine the appropriate classification category. These 
codes were altered as soon as the situation changed. 
Finally, the system had a real-time error prevention 
feature in which codes not recognized by the software 
were not permitted, ensuring accurate coding categories. 
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This feature is an important consideration when recording 
absolute behavioral frequencies in a natural environment 
(Skrtic k Sepler, 1982). 
Research Design 

The research design is shown in Figure 2 and 
describevi below: 



Basel ine 



Trainj ng & Feedback 



Group A 



Group B 





Intervention 1 | 
1 
i 




1 


Ii tervcntion 2 


\ 

1 


1 

1 2 3 j 4 5 6 


7 8 9 



Observations 

Figure 2 . Research Pes jgn 

Baseline ,. The baseline phase covered a period of six 
weeks, during which students were observed th -e times 
each. After each observation, the observers discussed 
the lesson with the teacher trainee and gave them a copy 
of their field notes. Field notes were written on a 
standard observation form which consisted of a blank 
piece of paper with the student's name, current date, 
class period and other identifying information at the 
top. This form was carbonless, and produced three copies 
of the field notes--one of which was given to the 
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classroom teacher and one to the field experience student, 
while the final copy remained with the observer. The 
suggested procedure for filling out this form was a brief 
anecdotal record and critique of what occurred while the 
observer was present in the classroom (see Appendix A). 
St'ident strengths were noted with a plus ( + ), and 
weaknesses with a minus (-) sign. This we.s the only form 
of feedback provided to all students during zhe baseline 
phase of the study. 

Intervention 1 : Group A ALT Training . Following zhe 
baseline phase, students in both groups were given a 
three-hour training session involving different content. 
Each training session was conducted by the investigator. 
Students in Group A received a review of recent research 
studies which have demonstrated that certain teacher 
behaviors produce increased student acader.ic gains. The 
concept of Academic Learning Time (ALT) was emphasized, 
and studies were reported which have supported the 
efficacy of such teach er behaviors as active instruction, 
reduction of transition time, and maintaining a high 
level of student engagement. 

Effective ALT teaching strate ies were then 
explained to Group A students in relation to the 
various observational categories on the STEEL observation 
system, previously discussed. Student questions 
concerning the research literature were clarified and 
overall skill improvement strategies uere suggested. For 
example, students were advised to try to increase their 

Er!c ijii 
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use of active insT:ruction and student engagement rates, 
and to try to decrease passive instruction, transition 
t?me, and student non-engagement. Anonymous examples 
were provided students using data from observations 
conducted on previous participants in the mildly 
handicapped program: the interrelationships between 
various types of teacher behavior and its subsequent 
effect on classroom student behavior were noted. An 
example of this type of training material is provided in 
Appendix B. 

One week later, prior to field observations. Group B 
students were also given a three-hour training session in 
"humanistic classroom management."' This topic was 
selected because it had an appropiate degree of face 
validity, yet was sufficiently distinct from the 
research-based training information. A series of 
videotaped vignettes were shown to the studeni:s. The 
investigator stopped the tape at appropriate poants to 
discuss the information and to assess studenc mastery of 
concepts presented on the tape. No mention was made of 
the ALT literature, the microcomputer-based observation 
system, or of the field observation procedures. 

Following these training sessions, each group of 
students participated in a six-week field placement. The 
observers provided computer-generated printouts and 
feedback as well as field note feedback to Group A. 
Group B received only field notes. Or^oviding Group A 
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with field notes in addition to tie computer feedback was 
aeemed necessary by the field experience instructor, who 
felt it important ro maintain a degree of normalcy for 
the students. Observers were blind to condition; 
differential feedback was based upon a list of student 
identification numbers. 

Int>^rvention 2i Group B ALT Training . -^rior to the 
second intervention phase, an additional training session 
was held. In this session, students in Group B were 
given training identical to that received by Group A 
students in the fall. Group A students were not given 
any additional training, but rather received individual 
printouts of their aggregated classroom teaching 
performance during the baseline and first intervention 
phases. Improvements in individual percentage scores in 
the areas of active and passive instruction and 
transition time as well as student engagement were noted. 
The investigator encouraged students in Group A to 
maintain or, if necessary, improve specific teaching 
behaviors during the second intervention phase. 
Computer Data Collect] on Procedures 

Instrument. The STEEL observa on system was used 
to collect data for this study. This observation system 
was microcomputer-based, i.e., resident in an Epson HX-20 
microcomputer, and utilized a real-time coding system 
which allowed electronic recording and data analysis oi 
teacher trainee behavior in classroom settings. 
Behaviors vere coded as the teacher trainee conducted a 
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lesson, then computer printouts showing actual amounts of 
time spent on various activities were provided as 
feedback, a technique identified as being a more 
effective evaluation strategy (Medley, Coker, & Soar, 
1984) . 

Classroom Observations . When the observers first 
entered a classroom, they moved quietly to a position 
where they could clearly observe tne teacher trainee and 
avoid distracting the class. Once situated, ,he observer 
turned on the computer and selected one of the five 
programmable- function keys. Function 2 ("COLLECT") 
engaged a softx^are program which prompted the observer to 
enter "header" information, including (a) the date, (b) 
their observer number, and (c) the identification number 
of the student whom they were observing. After this 
information was entered, the computer prompted the 
observer to select the most appropriate beginning code 
for each observational category. 

For the observers' benefit, an abbreviated lirt of 
codes and checklist items was located on the left side of 
the computer. The observer first selected the 
appropriate group structure code which identified the 
portion of students in the class for whom the teacher 
trainee was responsible. Ii they were responsible for 
only a small group, the computer next prompted them to 
enter the number of students in that partial group. If 
the teacher trainee was instructing a small group but 
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was also responsible for monitoring other students, the 
observer was prompted to indicate the size of the 
monitored group. Next, they enrered a code identifying 
the educational activity, i.e., "active instruction", 
"passive instruction", "transition", or "can't tell". 
Fourth, the teacher instructional behavior was recorded. 
Finally, observers recorded the student behavior, both in 
the target and monitored (if applicable) groups. These 
"start up" codes had to be entered before further coding 
could occur. After this initial coding was completed, 
codes were changed as either teacher, i.e., field 
experience student or classroom student beha^^ior changed. 

Using real-time observation coding procedures, 
observers coded the actual duration of each behavior in 
terms of seconds. While a code from a particular 
behavior category was in effect, time was incremented jn 
that category by the computer's in^,ernal clock. This 
condition remained in effect until observers entered a 
new behavior code in that category. Wher a code was 
changed, time began to be incremented in the new behavior 
category. This procedure was followed for each 
observable teacher and/or st..atnt behavior until the 
observation session was completed. To end the session, 
observers entered a "00" code, which turned off the 
coniputer's internal clock. Then, the observer was given 
an option to actually halt or to resume coding. If the 
observer indicated that they did not wish to resume 
coding, i.e., the observational period was complete, they 
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pressed the "N" (No) koy. 

At this point, the computer prompted observers to 
proceed to the checklist ratings by selecting Function 3 
("CHECKLIST"). The observer was prompted to rate a 
student's performance on any of a total of 68 checklist 
items. When the observer selected a given item number, 
it was displayed on the computer screen along with the 
rating scale, and the observer entered the appropriate 
value, whether 1 (Low), 2 (Medium), or 3 (High). When the 
entire observation was finished, the observer selected 
Function 4 ("REPORT"). This produced a printed report of 
the aggregated category totals, expressed both as a 
percentage of total observation time and actual minutes. 
Observers then shared and discussed the observation results 
with the teachsr trainee (see Appendix C) . 

All observational data were aggregated in the 
computer's internal memory. It remained there until the 
observer selected Function 5 ("SAVE"), which moved the 
observation data from internal memory and stored them on 
a microcassette tape, also included in the Epson HX-20. 
With the observational data sav._d on cassette, the 
computer was ready for the next observation. When the 
microcassette tapes were full, the data were transferred 
to the student's larger database located on a mainframe 
coinputer . 



ERIC 



93 



ERIC 



38 

Observer Training 

L a boratory t raini . n ^. Each observer pax-ticipated in 
a day-iong training program provided by the investigator 
regarding the computer observation system in addition to 
a general 20 to 30 hour orientation in supervisory 
techniques provided by the field experience instructor, 
At the beginning of the 1985-86 academic year, an eight- 
hour training session was held with the .hree supervisor/ 
observers. Training included a general presentation and 
discussion of (a) the ALT research literature and how 
it related to the STEEL project, (b; the STEEb 
classifications, categor.-i es , and checklist items, (c) the 
STEEL observation system and, (d) the use of the Epson 
HX-20 portable microcomputer. 

Afte- thoroughly cUscussing the observation system 
and behavioral categories, the supervisors practiced coding 
videotaped segments of classroom interactions. The 
observers coded three different classroom vignettes 
demonstrating behaviors included in the observation 
system. Interobserver agreement information was taken 
for each of these coding sessions and used as feedback to 
observers to consensually validate the observation 
categories. Reliability was estimated using the Flanders 
reliability coefficient formula (Flanders, 1967). This 
form..'la was selected because it xs <^ more appropriate 
reliability coefficient for situations in which the 
intended unit of analysis is category proportions (Frick 
& Semmel, 1978). Reliability coefficients ranged from 
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.86 to .91. Since an interobserver agreement coefficient 
of .75 is generally considered acceptable (Johnson & 
Bolstad, 1973), observers were judged to be coding the 
videotapes reliably, and training was concluded. 

Field calibration . Interobr.-'^rver agreement checks 
were conducted five times throughout the course of the 
study. The investigator would meet with individual 
observers in field classrooms and serve as the criterion 
coder. Observers were instructed to emphasize accuracy 
rather than interobserver agreement (Boykin & Nelson, 
1981) . 

Observer 12 was visited by the investigator twice 
during the month of November. The first observer 
agreement check yielded a Flanders coefficient of .73. 
The criterion observer (investigator) and Observer 12 
discussed discrepancies between observation results and 
problematic behavioral codes were clarified. A second 
field check conducted with this individual two days 
later obtained an interobserver agreement rating of .85, 
which was judged acceptable. 

Observer 14 also underwent a field reliability 
calibration with the inve-. t igator . Because some initial 
field data generated by Observer 14 was questionable, 
i.e., the codes generated in one category did not agree 
with other observational codes, it was discarded and an 
additional two hours of laboratory training was provided. 
Following this training, Observer 14 produced an initial 



field agreement check of .80, and a second check, 
conducted the following week, of .94. At this time, 
the data generated by Observer 14 was judged to be 
reliable and was subsequently included in the database. 

Observer 13 was visited once in the schools during 
the first semester. This individual had achieved the 
highest interobserver agreement ratings during the 
laboratory training period. As Observer 13 obtained a 
Flanders coefficient of .98 on the first field check, no 
further field calibrations were performed. 

To ensure that there was no observer drift over the 
Christmas break, field observer agreement data was again 
collected in January of 1986. Interobserver agreement 
scores of .88 (Observer 14), .95 (Observer 12), and .97 
(Observer 13) were obtained. As these checks far 
exceeded acceptable levels of agreement (Johnson & 
Bolstad, 1973), no further field calibrations were 
conducted after this time. 
Attitude Scale 

A twenty-four item attitude scale, the Student 
Evaluation of Observatio^i System (SEOS), was developed 
specifically for the study. Items were written by the 
investigator and submitted to two knowledgable external 
reviewers for content validation. Items on the scale 
asked students to rate the usefulness and clarity of both 
types of feedback, the quality of explanations, and the 
degree of threat presented by both types of evaluative 
feedback (see Appendix D) . 



The SEOS was administered to determine whether there 
were differences in the attitudes of the two groups (A 
and B) regarding the two types of observational feedback 
as a result of the differential training. The scale was 
administered twice to students involved in the study. 
The first administration occurred at the end of the first 
intervention phase. At this time, Group A students had 
received the ALT training/computer feedback for three 
observations while Group B students had received only 
field notes. Responses to the scale were ostensibly 
anonymous, but students were tagged according to which 
section of the reading methods course they were enrolled 
in to determine their status in the study. Students were 
asked to rate 18 items on the attitude scale relating to 
both the computer and field note feedback systems on a 
scale of 1 (Strongly Agree) to 5 (Strongly Disagree). 
Six open-ended questions were also included, asking for 
student reactions to both forms of feedback. 

The attitude scale was administered for the second 
time at the end of the second intervention phase. At 
this time, all students had been given the ALT training. 
Group B students had received computer-based feedback for 
three observations. Group A students for six observations 
Students responded to the same items, and responses were 
once again collected anonymously. This enabled an 
analysis of student attitude change toward he 
microcomputer-based observation system as a function of 
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the information presented at the second training session. 
Data Analysis 

Analysis of the observational data was conducted in 
the following manner: within each phase of the study, 
individual students' percentages of observed time in the 
categories of active and passive instruction, transition, 
and active and passive classroom student engagement were 
aggregated for each phase of the study. Then, these 
totals were divided by the number of times the student 
was observed during that phare, thus providing a mean 
percentage rate for each student. This was necessary 
because observation periods varied in length, so 
comparing students by observations alone would not 
provide an accurate measure. The final step involved 
combining the individual student means and dividing that 
total by the number of students in each group, i.e., A or 

In this manner, group means were obtained for each 
condition in the study. These means were then compared 
using T-tests in order to evaluate the difference between 
the groups as a result of training effects. The 
probability level for type-I errors was set at .05. 
Using this procedure, main effects between the groups in 
the categories of active and passive instruction and 
transition as well as concomitant classroom student on- 
task behavior over all phases of the study (Baseline - 
Intervention 1 - Intervention 2) were examined. 

All mean scores were rounded to the nearest hundred. 
Percentages did not always equal 100 per cent because 
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incidental amounts of observation ci.-'e were coded in the 
categories of Null (no instruction occurring in the 
classroom) and Can't Tell (observer unable to immediately 
ac^ertain teacher trainee/classroom student behavior). 
First, a between groups comparison was performed, where 
the performance of Group A was compared with the 
performance of Group B for each phase of the study, 
resulting in three separate comparisons (Baseline - 
Intervention 1 - Intervention 2). Then, each group was 
examined individually using a repeated measures design to 
determine the degree of change which occurred within 
groups over the phases of the study, resulting in two 
major comparisons for each group. For Group A, 
significant changes were expected to be observed between 
the Baseline and Intervention 1 phases. To examine 
whether any observed changes were maintained over time, 
the Intervention 1 mean score was then compared with the 
Intervention 2 mean. For Group B, the Baseline - 
Intervention 1 comparison was not expected to be 
significant, while the Intervention 1 - Intervention 2 as 
well as the Baseline - Intervention 2 comparisons should 
show significant differences. 

Attitude scale data were analyzed using the 
Kolmogorov-Smirnov Two-Sample Test (Siegel, 1956). This 
nonparametric test was selected because the attitude 
scale utilized a 5-point rating system on which students 
rated responses from 1 (Strongly Agree) to 5 (Strongly 
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Disagree), Since this rating system invol/ea an ordinal 
scale, it was necessary to examine the data using a 
statistical procedure which makes only minimal 
assumptions about the form of the underlying 
distributions of the data- The Ko Imogorov-Smirnov Two- 
San,ple Test measures the homogeneity of the distribution. 
It allows one to determine whether the attitudes of one 
group were significantly different from those of the 
second group- Examining the attitude scale data in this 
manner was intended to identify differences in student 
attitudes which occurred as a result of the differential 
training provided. Additionally, student responses to 
the open-ended questions were examined using naturalistic 
iTiethodology (Cuba & Lincoln, 1981), 
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RESULTS 

From the data yielded by the microcomputer-based 
observation instrument, three teache.- and two pupil 
behaviors wore selected as dependent variables: 

(a) the teacher trainees' levels of time spent in active 
instruction, passive instruction, and in transition, and 

(b) classroom pupils* active and passive engagement rates. 

Other dependent variables examined in this study 
include student ratings on 20 computer checklist items 
measuring general academic and behavior management 
strategies which were evaluated on a three-point see le , 
and student responses to 18 attitude scale items which 
were measured on a five-point scale • 
Between Groups Analyses 
Observational Data: Baseline Phase 

Effects of the differential training on three 
te&cher behavior and two student engagement variables 
were examined by comparing the two groups' performance by 
phase. These comparisons are shown in Tables 2 and 3, 



Insert Tables 2 and 3 about here 



Teacher trainee instructional behaviors . In the 
category of active instruction. Group A produced a mean 
of 66 per cent during the baseline period, while students 
in Group B had a mean of 60 per cent. The t-test results 
indicated no significant differences between the groups 
regarding their use of active instructional behaviors 
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Table 2 

Group Means and Standard Deviations for De^pendent 
Var 3 ables By Phase 

Baseline Int. i int. 2 



Active 
Instruction 
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Variable 

by Group M SD M SD M SD 



A 66.7 23.9 94.7 5.2 86.5 11.8 

B 59.8 27.6 69.8 25.2 94.5 7.9 

Passive 
Instruction 

A 23.5 24.3 3.0 4.3 7.1 10.0 

B 25.5 17.7 19.9 22.1 4.5 7.7 

Transition 

A 5.3 4.9 1.2 1.7 4.3 4.1 

B 12.1 15.8 8.9 8.9 .88 1.3 

Active 

Student 

Engagement 

A 59.1 22.8 81.4 15.5 69.1 13.7 

B 53.4 25.7 70.6 14.2 64.0 15.8 

Passive 
Student 
Engagement 

A 24.0 24.7 11.8 13.4 22.8 11.2 

B 34.5 28.6 19.9 10.9 27.8 14.8 
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Table 3 

Between Groups t-Values for Dependent Variables Bv phase 
(separate variance estimates are reported) 



Variables 
by Phase 



Baseline 

Active Instruction 24.89 .60 

Passive Instruction 21.85 -.24 

Transition 15.63 -1.52 

Active Student Engagement 24.95 .61 

Passive Student Engagement 24.88 -1.02 

Intervention 1 

Active Instruction 14.18 3.61 

Passive Instruction 14.08 -2.80 tt 

Transition 13.98 -3.20 

Active Student Engagement 24.37 1.89 

Passive Student Engagement 23.14 -1.73 

Intervention 2 

Active Instruction 20.88 -2.05 

Passive Instruction 22.51 .75 

Transition 14.16 2.87 ** 

Active Student Engagement 24.90 ,89 

Passive Student Engagement 24.02 -.99 

*n<.05. **i2<.01. 
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during this phase. For the three observations sessions 
comprising the baseline phase, Group A had a mean of 24 
per cent and Group B an average of 26 per cent in the 
category of passive instruction- The t-test comparison 
revealed no significant differences between the groups in 
their use of this instructional strategy during the 
basel ine phase • 

In the transition category, Group A spent a mean 
of 5 per cent of available time on transitional 
activities during baseline, while Group B averaged 12 per 
cent. Although the observed means of the groups were 
more discrepant regarding time spent in transitional 
activities, the difference did not reach a statistically 
significant level • 

Classroom student behavior . The data indicate that, 
during baseline. Group A maintained a mean of 59 per cent 
active student engagement, while Group B had a mean of 53 
per cent. No significant difference was observed between 
the groups. During this same phase, Group A had an 
observed mean of 24 per cent in the category of passive 
instruction, while students in Group B maintained an 
average of 35 per cent in the same category, again 
nonsignif icant , 

Thus, th'.i first hypothesis, that no significant 
differences • uld be observed between the groups in the 
amounts of t ; devoted to active instruction, passive 
instruction, and transitional activities during the 
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baseline phase, was accepted. Likewise, the second 
hypothesis, that there would be no significant 
differences between the groups in observed levels of 
student engagement during the baseline phase was also 
accepted • 

Observational D ata; Intervention 1_ 

Teacher trainee instructional behaviors . During 
this phase of the study, the mean in the category of 
active instruction for teacher trainees in Group A 
increased to 95 per cent, while Group B trainees 
increased the mean percentage of active instruction to 70 
per cent. A highly significant difference (p < ,002) was 
observed between the groups regarding their use of active 
teaching behaviors during this phase. In the category of 
passive instruction, Group A had an observed mean of 
three per cent, while Group B had a mean of 20 per cent. 
Again, a significant difference {p < .012) was observed 
between the groups. 

The amount of available time spent in transitional 
activities by students in Group A was reduced to one per 
cent during the first intervention phase, while Group B 
maintained an average of nine per cent. Thus, students 
in Group A spent significantly less time (p < .005) in 
transitional activities than students in Group B during 
the first intervention phase. 

Classroom student behavior . The mean active 
student engagement percentage score for teacher trainees 
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in Group A during the first intervention phase was 81 per 
cent, while Group B had an observed mean of 71 per cent. 
While differences between the groups in this category 
approached significance (p < .07), it did not meet the 
predetermined level of .05. 

Students in Group A had an observed mean of 12 per 
cent in the category of passive student engagement. 
During the same time, students in Group B maintained a 
mean of 20 per cent. Again, differences between the 
groups in this category approacheH (p < ,09), but did not 
reach significance . 

The third hypothesis, that the information regarding 
effective teaching practices in combination with the 
computer-generated feedback would result in statistically 
significant differences between the groups in favor of 
Group A for active teacher instruction during the first 
intervention phase was supported. The fourth hypothesis, 
that the ALT training/feedback would significantly affect 
student engagement means in favor of Group A during the 
first intervention phase was rejected. As noted, 
differences between the observed mean student engagement 
scores for groups A and B approached, but did not meet, 
the predetermined level of statistical significance. 
Observational Data; Intervention 2 

Teacher trainee instructional behaviors . During 
this phase of the study, students in both groups were 
again observed three times. Students in Group A obtained 
an average of 87 per cent active instruction while Group 
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B students increased their mean in this category to 95 
per cent. Thus, the groups became more similar in their 
observ se of active instructional behaviors following 
the seuond training, and no significant differences were 
observed. 

In the category of passive instruction, students in 
Group B reduced their observed mean to five per cent 
during the second intervention phase, while students in 
Group A obtained an average mean score of seven per cent. 
Again, this indicates that the groups became more similar 
in their use of this instructional strategy, with no 
significant differences observed. 

Group B students reduced their mean percentage score 
in the category of transition to approximately onj per 
cent while Group A had an observed mean of four per cent 
This discrepancy between group means for the category 
of transition resulted in a significant difference 
between the two groups during this phase of the study. 

Classroom student behavior . The observed active 
student engagement means of both groups were highly 
similar during the second intervention phase. Group B 
studenrs had a mean of 64 per cent active student 
engagement, while Group A had a mean of 69 per cent, a 
nonsignificant difference. 

In the category of passive student engagement, the 
groups were again more homogeneous. Group B had a mean of 
28 per cent, while Group A had an observed mean of 23 per 
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cent. No significant differences were observed between 
the groups in their levels of passive student engagement 
during the second intervention phase. 

Based upon the above results, the fifth hypothesis, 
that there would be no significant differences between 
the groups in the categories of active instruction and 
student engagement during the second intervention phase 
ras accepted. However, the significant difference 
observed between the groups in the category of transition 
tempers the acceptance. 
Checklist Item Data 

Twenty checklisc items were selected and used to 
examine the effects of differential training/feedback on 
teacher trainee performance. These items were chosen 
because they were judged to have the strongest 
relationship with the concept of ALT, Items were rated 
by the observers on a three point scale, with l=Low, 
2=Medium, and 3=H3gh, Group means and standard 
deviations are listed in Table 4 for each of the three 
phases of the study. Table 5 shows the between groups 
comparisons by phase, witft statistically significant 
changes noted. 



Insert Tables 4 and 5 about here 



The results demonstrate that, while both groups 
showed improvement on their checklist ratings over the 
course of the study, few significant differences between 
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Table 4 

Between Groups. Means and Standard Deviations on 
Checklist Items by Phase 



Checklist 

Item Group 



01 



A 
B 



Baseline 



M 



SD 



1.7 .52 
1.4 .59 



Int . 1 



M 



SD 



2.3 .71 
1.6 .52 



Int. 2 



M 



2.7 
3.0 



SD 



47 
0 



02 



A 
B 



1 . 9 



39 



2.0 .71 



2.8 .42 
2.1 .33 



2.8 
2.9 



.44 
.33 



03 



A 
B 



2.6 
2.4 



.52 
.52 



3.0 
2.7 



0 

.48 



2.9 
2.8 



. 33 
.45 



04 



A 
B 



2.5 
2.5 



.67 
.69 



2.9 
3.0 



. 29 

0 



3.0 
2.9 



0 
33 



05 



A 
B 



2.4 
2.5 



.67 
.69 



3.0 
2.6 



0 
53 



3.0 0 
2.9 .33 



06 



A 
B 



2.5 .53 
2.8 .44 



3.0 
2.6 



0 
53 



3.0 
3.0 



0 
0 



07 



A 
B 



2.7 .48 



2.8 .44 



3.0 0 
2.9 .33 



3.0 0 
2.9 .33 



08 



A 
B 



2.4 
2.6 



.53 
.52 



2.9 
2.7 



. 33 
.50 



3.0 
3.0 



0 
0 



( table continues ) 
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Table 4 

Between Groups Means and Standard Deviations on 
ChecklisL Items by Phase 



Checklist 

Item Group 



09 



A 
B 



Basel ine 



M SD 



2.5 .52 



2.5 



69 



Int. 1 



M SD 



3.0 0 
2.7 .50 



Int. 2 



M SD 



3.0 0 
2.7 .50 



14 



A 
B 



2.4 
2.5 



.67 
. 78 



2.7 .68 

2.8 .42 



2.9 
3.0 



32 
0 



19 



A 
B 



2.2 
2.6 



. 58 
.67 



2.9 .29 
2.6 .52 



2.9 .33 
3.0 0 



20 



A 
B 



2.3 
2.6 



.49 
. 79 



2.S 
3.0 



35 
0 



3.0 
3.0 



0 



22 



A 
B 



2.4 
2.6 



, 51 
65 



2.9 
2.9 



29 
38 



3.0 
2.9 



0 
38 



30 



A 
B 



2.3 
2.6 



. 46 
. 51 



2.9 
2.9 



35 
39 



3.0 
3.0 



0 
0 



50 



A 
B 



2.1 
2.2 



. 99 
1.0 



2.8 .41 



2.5 



82 



2.8 



39 



2.6 .52 



51 



A 
B 



2.2 .79 
2.4 .79 



2.6 .52 
2.3 .71 



2.6 .74 
3.0 0 



( table continues ) 
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Table 4 

Between Groups Means and Standard Deviations on 
Checklist Items by Phase 



Checklist 

Item Group 



56 



A 
B 



Baseline 



M 



1 . 7 
2.2 



SD 



.76 
.83 



Int. i 



M 



SD 



2.7 

2.8 



.49 
,46 



Int. 2 



M 



3.0 
2.8 



SD 



0 

.50 



55 



90 



A 
B 



2.4 
2.5 



.67 
.66 



2.6 
2.8 



.70 
68 



3.0 
2.8 



0 

.46 



92 



A 
B 



2. 1 
2.4 



,74 
,52 



3.0 
2.7 



0 
49 



3.0 
2.8 



0 
45 



95 



A 
B 



2 . 3 
2.8 



.50 
.45 



3.0 
3.0 



0 

0 



3.0 
2.8 



0 
45 



Note 



01 Conveyance of importance of curriculum content 

02 Conveyance of importance of pupil performance 

03 Use of transition time 

04 Degree of task-orientation 

05 Organization of time 

06 Organization of presentation 

07 Organization of materials 

08 Sequencing of instruction 

09 Pacing of instruction 
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14 Clarity of directions 

19 Provision of feedback about pupil performance 

20 Provision for basic skill acquisition 
22 Provision for successful experience 

30 Educational relevance/soundness of instruction 

50 Awareness of classroom dynamics 

5 1 Clarification of behavioral expectations 
56 Consistency in use of behavior management 

techniques 

90 Degree of student task-orientation 

92 Level of group participation 

95 Degree of student task success 



the groups were observed. During the baseline phase, the 
groups were highly homogeneous regarding their checklist 
ratings • 

During the first intervention phase, the groups 
received significantly different ratings on only four 
items • One item reached significance during the second 
intervention phase* The higher ratings on these items 
were received following the ALT training sessions. These 
improvements were observed even though the rating scale 
had a very limited range (1 to 3) and was thus fairly 
insensitive to change. These results must be viewed 
tentatively, but are suggestive of improved teacher 
performance as a result of the ALT training, 

ERIC j ] 4 



Table 5 

P^^^e^n Groups t-Values for Checklist Items bji Phase 
Checklist 

Period Item df t 



01 


1 


, 10 


.83 


02 


1 


15 


-.48 


03 


1 


17 


.92 


04 


1 


22 


-.16 


05 


1, 


21 


-.63 


06 


1, 


18 


-1 .24 


07 


1, 


22 


-.36 


08 


1, 


18 


-.65 


09 


1, 


21 


0 


14 


1,22 


-.33 


19 


1, 


31 


-1 .63 


20 


1, 


18 


-.82 


22 


1, 


23 


-1 .04 


30 


1, 


17 


-1 .70 


50 


1, 


22 


-.31 


51 


1, 


21 


-.64 


56 


1, 


15 


-1 . 26 


90 


1 ,24 


-.17 


92 


1, 


17 


-.89 


95 


1, 


20 


-2 .00 



( table continues ) 
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Table 5 

Between Groups t -Values for Checklist Items by Phase 



Checklist 
Period Item df 



Int. 1 



01 


1,17 


2. 33 


02 


1 , 18 


3 . 92 


03 


1 , 18 


1 .86 


04 


1,21 


-.91 


05 


1 , 16 


2. 38 


06 


1 , 16 


2. 38 


07 


1 ,15 


.87 


08 


1 ,18 


1 .11 


09 


1,17 


1.88 


14 


1 ,19 


-.40 


19 


1,21 


1.82 


20 


1 , 14 


- . 93 


22 


1 , 18 


. 39 


30 


1,14 


.09 


50 


1 ,20 


1 . 32 


51 


1,17 


1.22 


56 


1,14 


-. 15 


90 


1 , 18 


-.57 


92 


1 , 13 


1.55 


95 


1 , 15 


0 



( table continues ) 
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Table 5 

Between Gt^^^u'^^c t-^^oT^^^o 4^^^ r-u^^t-i ^ -^4. to. , -l-^ . ^ 

Checklist 

Period Item df t 



01 


1 ,20 


-1 . 84 


02 


1,21 


-.69 


03 


1 ,13 


. 42 


04 


1 ,18 


1.11 


05 


1 ,18 


1 .06 


06 


1,17 


0 


07 


1 ,18 


1 .06 


08 


1 ,16 


0 


09 


1 ,18 


2.22 


14 


1,17 


-.89 


19 


1,17 


-.94 






0 


22 


1,14 


1 .07 


30 


1,14 


0 


50 


1,21 


1.21 


51 


1 : 16 


-1 .52 


56 


1,9 


1 .00 


90 


1,17 


1 .72 


92 


1,9 


1 .00 


95 


1 ,12 


1 .30 



. 05 . . 01 . . 001 . 
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Attitude Scale Data 

Analysis of items measured on the Student Evaluation 
of Observation System attitude scale are shown in Tables 
6, 7, and 8. To enhance the clarity of presentation, 
items were grouped into topical categories. Items 
pertaining to the field note feedback are presented 
first. Then, items pertaining to the microcomputer 
feedback system and finally, items relating to the 
computer checklist feedback are presented. Kolmogorov- 
Smirnov Z scores (Siegel, 1956) and associated 2-tailed 
probability levels are reported for each group of items. 
It should be noted that the items from each of the 
categories were interspersed on the original Student 
Evaluation of Observation System attitude scale (see 
Appendix D) , so the numbers used in the following tables 
do not necessarily correspond with the original numbering 
system . 
Table 6 

Between Groups Comparison of Field Note Items 

Item No. Administration 1 Administration 2 



1 


Z = l 


.23, 


p< . 10 


Z = 


.61, 


p< . «5 


2 


Z = 


.82, 


p< . 52 


Z = 


.41, 


p<l .0 


3 


z= 


. 82, 


p< . 52 


Z=l 


. 02, 


p< . 25 


4 


z= 


.41, 


p<l .0 


Z = 


.20, 


p<l .0 


5 


Z = l 


02, 


p< . 25 


Z= 


.20, 


p<l .0 
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Note • 

1, The supervisors were skillful in 
providing field note feedback, 

2, The field note feedback was helpful 
in improving my teaching skills, 

3, The field note feedback doesn't provide 
me with clear ideas of what "chings I 
need to do to improve my teaching, 

4, I feel comfortable with the supervisors 
using the field note method to evaluate 
my teaching • 

5, The field note feedback should be retained 
as part of the undergraduate field 
supervision program. 



It can be seen that student attitudes toward the 
field note feedback did not change between the two 
attitude scale administrations. In other words, the 
differential training provided to the groups did not 
affect their generally positive attitudes toward this 
form of feedback. Comments made by the students 
supported this finding, and will be discussed later. 
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Table 7 

Between Groups Comparison of Computer Observation Items 



Item No, 



Administration 1 



1 
2 
3 
4 
5 
6 
7 
8 



1- .20, p<1.0 

Z= .61, p<.85 

Z=2.04, p<.000 *** 

Z=1.84, p<.002 ** 

Z=1.43, p<.03 * 

Z= .61, p<.85 

Z=1.23, p<.10 

Z=2.04, p<.000 ** 



Administration 2 

Z=0, p<1.0 
Z=.20, p<1.0 
Z=.20, p<1.0 
Z=.41, p<1.0 
Z=.20, p<1.0 
Z=.41, p<1.0 
Z=.41, p<1.0 
Z=.20, p<1.0 



Note . 



1. 



2 . 



3 . 



4 . 



5. 



6 . 



The computer was always used 

durxng the observations. 

I always received the computer 

summary from the observers. 

The supervisors clearly explained 

the computer summaries of my 

teaching behaviors. 

I clearly understand the various 

categories on the observation system. 

The computer-based feedback was helpful 

in improving my teaching skills. 

Being evaluated by the computer observation 

system is a threatening experience. 
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1* I feel comfortable with supervisors 
using the computer observation system 
to evaluate my teaching, 

8, The computer olservation feedback 
should be retained as part of the 
undergraduate field supervision program. 

The groups showed significant differences on four 
of the eight attitude scale items involving the 
microcomputer-based observation system on the first 
attitude scale administration. These items were intended 
to measure the teacher trainees' professed level of 
understanding of the computer-based feedback and whether 
or not they perceived it as beneficial. Results indicate 
that there were clear differences in favor of Group A 
regarding both the value and usefulness of this type of 
performance f eedback , 

Thus, the first half of hypothesis number nine, that 
there would be a statistically significant difference 
between the groups in favor of Group A regarding 
attitudes toward the microcomputer-based observation 
system following the first training session, was 
accepted. 
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Table 8 

Between Groups Oomparison of Checklist Items 



Item No, 



Administration 1 



Administration 2 



1 
2 
3 
4 



Z=1.84, p<.002 ** 

Z= .82, p<.52 

Z=2.04, p<.000 ** 

Z=1.43, p<.03 * 



Z=.20, p<1.0 

Z=.41, p<1.0 

Z=.41, p<1.0 

Z=.41, p<1.0, 



Note , 



2. 



3. 



1. The supervisors were skillful in 
explaining the checklist feedback. 
The checklist feedback (High-Medium-Low) 
was helpful in improving my teaching skills. 
I feel the checklist feedback system 
too confusing to be useful. 
4. The checklist feedback system should be 
retained as part of the undergraduate 
field supervision program. 

After the first attitude scale administration, the 
groups displayed significantly different attitudes toward 
the majority of statements relating to the checklist 
items. On the second administration, i.e., after both 
groups had received the AI training, these differences 
disappeared. Thus, the second half of hypothesis number 
nine, that differences between the groups would be 
nonsignificant following the second ALT training, was 
also accepted. 
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Within Group Analyses 

The effect of the differential training/feedback 
upon the performance of individual groups over the three 
phases of the study was also evaluated. Differences 
in individual group means and standard deviations on 
dependent variables can be examined in Table 2. To gain 
further insight into the effects of the ALT training and 
feedback procedures, a repeated measures t-test design 
was employed to examine intragroup performance change 
and, where appropriate, maintenance across phases. The 
results are summarized in Tables 9 and 10. First, 
changes in Group A will be examined. 



Insert Table 9 about here 

Observational Data: Group A. Students in this 
group were expected to increase their level of active 
instruction during the first intervention phase. Their 
group mean increased from 66 per cent during baseline to 
^5 per cent during the first intervention phase, a 
highly statistically significant (p < .001) change. 

Because this change in the use of active 
instructional behaviors was so robust, the first part of 
hypothesis number six, that students in Group A would 
show a statistically significant difference over their 
baseline level in the category of active instruction was 
supported. 
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Table 9 

Within Group t~Values for Training Effects: Group A 



Variables by 
Comparison 

Period df 



Baseline to 

Intervention i 

Active Instruction 12 -4*11 *** 

Passive Instruction 12 3.04** 

Transition 12 3.06 ** 

Active Student Engagement 12 -2.57 * 

Passive Student Engagement 12 1.56 



Intervention X 

to Intervention 2 

Active Instruction 12 * 2.06 

Passive Instruction 12 -1.36 

Transition 12 -2.84 * 

Active Student Engagement 12 1.84 

Passive Student Engagement 12 -1.65 



*n<.05. **n<.oi. ***n<-ooi. 
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During the second intervention phase, the observed 
level of active instruction for Group A decreased slightly 
from the first intervention phase level, from 95 to 87 
per ce t. This decrease was not significant, indicating 
that students in Group A maintained their active 
instruction gains over time* 

Group A began the study with an average baseline 
mean of 24 per cent in the category of passive 
instruction. This decreased to three per cent following 
the first intervention, a significant (p < ,01) decrease 
in their use of this instructional strategy. The group 
mean increased during the second intervention phase to 
seven per cent, a nonsignificant change* Again, this 
indicates that students in Group A maintained their more 
effective teaching strategies during this phase of the 
study . 

In the category of transition, Group A had an 
average level of five per cent during baseline, 
indicating that they spent relatively little time in 
transitional activities. However thi s level decreased 
to one per cent following the first intervention phase, a 
statistically significant (p < .01) reduction. During 
the second intervention phase, their observed level 
increased to four per cent. Thus, the amount of time 
that students in Group A allocated to transitional 
activities during the second intervention phase 
approached baseline levels, i.e., the reduced transition 
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levels were not maintained over time. 

In the category of active student engagement, the 
mean percentage score of Group A students increased from 
59 to 81 per cent during the first intervention phase, a 
significant (p < .02) change. Thus, the second part of 
hypothesis number six, that students in Group A would 
show a significant increase over their baseline level of 
student engagement during the first intervention phase 
was accepted. In the second intervention phase, this 
mean decreased to 69 per cent, a nonsignificant change, 
while the obsei ved passive engagement mean increased. 

Group A had an observed baseline level of 24 per 
cent in the category of passive student engagement. 
During the first intervention phase, this level decreased 
to 12 per cent, a nonsignificant change. 
Between the first and second intervention phases, the 
mean group percentage in this category returned to 
baseline levels, or 23 per cent, again a nonsignificant 
change . 

Observational Data : Group B. The within group 
t.-test values for Group B are shown in Table 10. 
Examining the observation data from baseline to the 



first intervention phase in the category of active 
instruction showed that students in Group B increased 
their mean from 60 to 70 per cent. This increase in 



Insert Table 10 about here 
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Table 10 

Within Group t-Values for Training Effects: Group B 



Variables by 

Comparison 

Period 



Baseline to 

Intervention 1^ 

Active Instruction 13 -1.01 

Passive Instruction 13 .83 

Transition 13 ^qq 

Active Student Engagement 13 -2.16 * 

Passive Student Engagement 13 1.70 



Intervention 1_ 

to Intervention 2^ 

Active Instruction 13 -3.93 ** 

Passive Instruction 13 2.76 ** 

Transition 13 3.50 ** 

Active Student Engagement 13 1.00 

Passive Student Engagement 13 1.21 



*£<-05. **n<.01. 
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the use of active instruction did not reach a 
statistically significant level. 

After Group B received the ALT training/feedback, 
i.e., in the second intervention phase , a significant ( p 
< .002) increase was observed. As previously noted, 
Group B had a 70 per cent active instruction level during 
the first intervention phase. In the second intervention 
phase, this increased to 95 per cent. This indicates 
that students in Group B used significantly more active 
instructional behaviors, and that this in';irease was due 
to some condition other than chance improvement or 
history. The first half of the seventh hypothesis, that 
there would be significant increase in the mean active 
instruction level of Group B during the second 
intervention phase, was accepted. 

Students in Group. B had an observed mean of 26 per 
cent in the category of passive instruction during 
baseline. This decreased to 20 per cent during the first 
intervention phase, a nonsignificant difference. After 
these students received the ALT training/ feedback 
however, their average use of passive instruction 
decreased to five per cent, a statistically significant 
(p < .01) change, "not reaching the predetermined level of 
statistical significance . 

Group B had an observed mean level of 12 per cent in 
the category of transition during the baseline phase. 
After the alternate training in humanistic classroom 
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management, this decreased to 9 per cent, a 
nonsignificant change. Following ALT training, the 
observed group mean in this category was reduced to 
less than one per cent, a significant (p < .001) change. 
This indicates that, while these students were able to 
reduce their observed levels of transition as a result of 
other factors, i.e., experience, placement, or alternate 
training, it was not until they received the specific ALT 
training that a significant change in their use of this 
teaching strategy was observed. 

Group B had a mean level of 53 per cent active 
student engagement in the baseline phase. During the 
first intervention phase, this increased to 71 per cent, 
a significant (p < .05) gain. In the second intervention 
phase, the observed level dropped to 64 per cent, a 
nonsignificant change. Thus, ALT training appeared to 
have little effect on the teacher trainees' observed 
levels of active student engagement, and the second half 
of hypothesis number seven, that the student engagement 
level for Group B would show a significant increase 
during the second intervention phase, was not supported* 

Teacher trainees in Group B averaged 35 per cent in 
the category of passive student engagement during the 
baseline phase. This decreased to 20 per cent during the 
first intervention phase, a nonsignificant change. 
Following the ALT training, the observed mean in this 
category increased to 28 per cent, which also failed to 
reach the predetermined level of significance. Again, 
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training appeared to have little effect on passive 
student engagsnient . 
Checklist Item Data 

Group means and standard deviations on the 20 
checklist items can be examined in Table 4. The t-test 
results for determining change within groups across 
phases are presented in Tables 11 and 12. 



Insert Tables 11 and 12 about here 

These results indicate that Group A showed 
significant improvement on 15 items from baseline to the 
first intervention phase, while Group B improved 
significantly on only two items during the same time 
period. However, Group B showed significant improvement 
on five checklist items during the second intervention 
phase, while Group A students maintained their higher 
mean ratings. These results allow acceptance of the 
eighth hypothesis, that student ratings on the computer 
checklist items would show improvement as a result of 
training . 
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Table 11 

Within Group t-Values for Checklist Items: Group A 
Comparison Checklist 

Period Item df t 



01 


1,17 


-1.50 


02 


1,20 


. 17 


03 


1,15 


1 .00 


04 


1 ,20 


-.96 


05 


1,15 


0 


06 


1,14 


0 


07 


1,14 


0 


08 


1,15 


-1.00 


09 


1, 16 


0 


14 


1,17 


-.85 


19 


1,18 


.20 


20 


1,7 


-.48 


22 


1,17 


-.81 


30 


1,13 


-1.00 


50 


1,20 


-.09 


51 


1,15 


-.08 


56 


1,8 


-1.14 


90 


1,17 


-1.81 


92 


1,9 


0 


95 


1,13 


0 



*£<.05. **£<.01. ttt^<. 001. 
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Tcible 12 

Within Group t-Values for Checklist Items: Group B 



Comparison 
Period 



Checklist 
Item 



df 



Baseline 
to 

Int. 1 



01 


1,10 


-.75 


02 


1,15 


-.43 


03 


1,15 


-1.37 * 


04 


1,18 


-2.09 * 


05 


1,17 


-.04 


06 


1,15 


.97 


07 


1,19 


-.69 


08 


1,16 


-.29 


09 


1,17 


-.44 


14 


1,20 


-1.24 


19 


1,19 


-.06 


20 


1,11 


-1.44 


22 


1,17 


-.90 


30 


1. 15 


-.99 


50 


1,21 


-.59 


51 


1,17 


.48 


56 


1,14 


-1. 58 


'90 


1,19 


-1. 10 


92 


1,12 


-1.30 


95 


1,17 


-1.55 



( table continues ) 
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Table 12 

Within Group t-Values for Checklist Item s: Group B 
Comparison Checklist 

Period Item df t 



01 


1 ,15 


-8.47 *** 


02 


1 ,15 


-4.95 *** 


03 


1, 12 


-.39 


04 


1 , 16 


1.06 


05 


1, 15 


-1.60 


06 


1, 15 


-2.53 ** 


07 


1,15 


0 


08 


1,14 


-1.88 


09 


1, 15 


0 


14 


1 ,15 


-1.33 


19 


1 ,15 


-2.18 * 


20 


1,5 


0 


22 


1,11 


0 


30 


1 , 18 


- . 92 


50 


1,18 


-.48 


51 


1,14 


-3.20 ** 


56 


1,9 


0 


76 


1, 12 


-1.91 


90 


1, 14 


.10 


92 


1,9 


-.31 


95 


1,10 


1.30 



*n<,05. **e<.oi. 001. 
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Attitude Scale Data 

Comparing changes within groups on attitude scale 
items over the three phases of the study would detect 
change in students' opinions of the different observation 
systems as a function of training. According to the 
original hypothesis, Group A, which received the 
research-based training prior to the first attitude scale 
administration, should show little change whereas Group B 
should show significant change in their attitudes toward 
the computer-based feedback system between the first and 
second intervention phases. For purposes of presentation, 
the attitude scale items have been grouped categorically, 
with field note item analyses presented first, followed 
by the computer feedback, then checklist item analyses. 
Tables 13 through 15 show Kolmogorov-Smirnov Z scores and 
the associated 2-tailed probability levels for items in 
each group. The first analysis involves field note 
items. Results are presented in Table 13. 



Insert Table 13 about here 



It can be seen from the data that there were no 
significant differences in the attitudes of Groups A or 
B regarding the field note feedback. Only the fourth 
item approached statistical significance for Group A, 
involving their perceived degree of comfort with the 
use of field notes as an evaluative tool. This finding. 




Table 13 

Analysis of Within Group Change in Attitudes Across 
Phases ; Field Note Items 



Comparison Period 
Group Item No. int. 1 to Int. 2 

Z= .43, p<.99 
Z=1.06, p<.21 
Z= .64, p<.81 
Z=1.28, p<.08 
Z= 0, p<1.00 



2 
3 
4 
5 



Si Z= .94, p<.34 

2 Z= .94, p<.34 

3 Z= .71, p<.7r 

4 Z= .47, p<.98 



Note 



5 



Z= .71, p<.70 



The supervisors were skillful in 

providing field note feedback. 

The field note feedback was helpful 

in improving my teaching skills. 

The field note f> idback doesn't provide 

me with clear ideas of what things I 

need to do to improve my teaching. 
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4. I feel comfortable with the supervisors 
using the field note method to evaluate 
my teaching. 

5. The field note feedback should be retained 
as part of the undergraduate field 
supervision program. 

albeit nonsignificant, indicates that students in Group A 
felt more satisfied with the field note feedback during 
the first intervention ph^se, and less satisfied during 
the second. 

Group attitude changes regarding the computer 
observation system feedback are presented m Table 14. 



Insert Table 14 about here 



Group A had only one significant difference across 
both attitude scale adminis tratio ^. Their responses to 
item number two indicated that they did not always 
receive computer summaries from the supervisor/observers 
during the first phase, but always received them during 
the second phase. This differential response pattern was 
unexpected, but probable re-^sons for its occurrence will 
be discussed . 

The most noticeable attitude change occurred in 
Group B, whose responses to six of the eight items were 
statistically significant. This indicates that the 




Table 14 

Analysis of Within Group Change in Attitudes Across 
Phase SI Computer Observation System Items 
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Comparison Period 



Group Item No, 



Int. 1 to Int, 



B 



1 


Z = 


.85, 


P< 


.46 


2 


Z=l 


.49, 


P< 


.02 * 


3 


z= 


.21, 


p<l .00 


4 


z= 


.43, 


P< 


. 99 


5 


z= 


.21, 


p<l . 00 


6 


z= 


.64, 


P< 


.81 


7 


z= 


.43, 


P< 


.99 


8 


z= 


.21, 


p<l .00 


1 


Z=l , 


,89, 


p<. 


002 ** 


2 


Z=l , 


,89, 


p<. 


002 ** 


3 


Z=l . 


41, 


p<. 


04 * 


4 


Z=l . 


41, 


p<. 


04 * 


5 


Z=l . 


65, 


p<. 


009 ** 


6 


z= . 


47, 


p<. 


98 


7 


z= . 


94, 


p<. 


34 


8 


Z=l . 


41, 


p<. 


04 * 



Note . 



1 . The computer was always used 
during the observations. 
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2. I always received the computer 
summary from the observers. 

3. The supervisors clearly explained 
the computer summaries of my 
teaching behaviors, 

4. I clearly understand the various 
categories on the observation system. 

5. The computer-based feedback was helpful 
in improving my teaching skills, 

6. Being evaluated by the computer observation 
system is a threatening experience. 

7. I feel comfortable with supervisors 
using the computer observation system 
to evaluate my teaching. 

8. The computer observation feedback 
should be retained as part of the 
undergraduate field supervision program. 

attitudes of students in Group B toward the computer 
observation system became much more positive following 
the ALT training session. While their attitudes toward 
the computer feedback were overwhelmingly negative on the 
first attitude scale administration, this changed during 
the second intervention phase. The results clearly 
indicate that attitudes toward the microcomputer-based 
feedback becam uch mors positive following training. 

Student responses regarding the checklist items are 
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presented in Table 15. 



Insert Table 15 about here 



Again, the data indicate no significant change in 
the attitudes of Group A between the two intervention 
phases. Their responses were highly positive on both 
administrations of the attitude scale. Group B, on the 
other hand, altered their opinions on three of the four 
items relating to the checklist feedback. Before 
receiving the ALT training, they did not feel the 
checklist should be retained as part of the undergraduate 
field supervision program; after training, they did. 
Thus, this training appears to have positively affected 
their opinions regarding this form of feedback. 

These results unequivocably show that the ALT 
training/feedback influenced students' opinions toward 
the computer-based feedback in a positive manner. As 
expected. Group A maintained their positive attitudes, 
for they had received research-based training and 
feedback prior to the first attitude scale administration. 
Their attitudes regarding the computer-based and 
checklist feedback also showed no deterioration over 
time. At the same time, their positive attitudes toward 
the field note feedback were maintained. 

On both attitude scale administrations, students in 
Group B were consistent in their attitudes toward the 
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Table 15 

Analysis of Within Group Change in Attitudes Across 
Phases; Checklist Items 

Comparison Period 
Group Item No. int. 1 to Int. 2 



A 1 Z=.21, p<1.00 

2 Z=.o4, p<.81 

3 Z=.21, p<1.00 

4 Z=.21, p<1.00 

B 1 Z=1.41, p<.04 * 

2 Z=1.18, p<.12 

3 ^ Z=1.41, p<.04 * 

4 Z=1.41, p<.04 * 

Note . 

1. The supervisors were skillful in 
explaining the checklist feedback. 

2. The checkliiit feedback (High-Medium-Low) 
was helpful in improving my teaching skills. 

3. I feel the checklist feedback system 
too confusing to be useful. 

4. The checklist feedback system should be 
retained as part of the undergraduate 
field supervision program. 
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field notes, inflicating that it was a useful form of 
feedback both times. However, on the computer-based and 
checklist feedback items, there were significant 
differences observed in their responses. The attitudes 
of students in Group B became significantly more positive 
following the ALT training session. This indicates that 
the training played a major role in altering their 
perceptions regarding the value of this form of feedback. 
This same pattern was also observed in their attitudes 
toward the usefulness of the checklist feedback. 

The available attitude scale data clearly indicate 
that the ALT training was instrumental in altering 
student reactions to the microcomputer-based feedback. 
Students involved in the study were also asked to respond 
to six open-ended questions regarding both types of 
feedback, and their responses will now be discussed. 
Attitude Scale Comments 

In addition to evaluating responses to the different 
forms of feedback on the quantitative, i.e., rating 
attitude scale, students were also asked to react to six 
open-ended questions. Their responses were analyzed in a 
manner designed to identify concerns and issues and to 
assess values (Cuba & Lincoln, 1981). 

The first question asked students what they liked 
about the field-note feedback. Responses were 
categorized into three areas: Pei.^sonal Comments, 
Specificity, and Constructive Criticism. Personal 
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Comments was the largest category, with 18 responses. 
One example included felt this allowed the observer 
to say exactly how she/he felt about something." 
Comments indicated that students found the narrative 
portion of the field note feedback most helpful because 
it discussed specific events that occurred during the 
observation; they were easily able to relate the feedback 
to those events. There were also criticisms of this form 
of feedback, however. For example, one student commented 
"It was helpful in that it let me know my goods and bads 
but it didn't give me ideas of how to improve." Also, a 
student stated that "... Sometimes the evaluator would 
mearly (sic) write down what I had done which was not 
helpful. My opinion of the field note depended on the 
competancy (sic) of the evaluator." Thus, students did 
not always feel that field notes gave specific direction 
for improvement and some perceived this form of feedback 
as helpful only when the observer's opinion was valued* 

The aspect of Constructive Criticism was also deemed 
important by some students, as evidenced by the inclusion 
of 11 comments in this category. This constructive 
criticism, when provided, was used by students to improve 
their teaching. A representative comment for this 
category was: "The field note feedback proved useful when 
suggestions and other options were included along with my 

evaluation. always added constructive comments and 

ideas for me to try. also would explain and comment 

on my performance and how to improve and areas of 
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concern • " 




A third category, Specificity, contained 12 




comments, including "The supervisor was able to be 




specific m her own words. The computer doesn't allow 




that," and also "It was specific. This part of the 




feedback informed me of particular aspects of my lessons 




that were not mentioned on the computer observation or 




checklist feedback." Thus, the observer's ability to 




relate field note feedback to specific parts of the 




lesson was valued by students. 




From the above comments, it appears that the 




students place a great deal of importance on the more 




personal aspect of the field notes. This form of 




feedback allows them to place their lesson(s) in context, 




with comments specifically related to observed events. 




It should be emphasized, however, that this is perceived 




as helpful only when the observer's opinion is valued. 




When the feedback was merely a reiteration of the lesson 




and/or when it failed to provide specific suggestions for 




improvement, field note feedback failed to meet the 




perceived professional development needs of the teacher 




trainees. 




A second item asked students what they disliked 




about the field note feedback. Comments fell into three 




major categories: Quality, Subjectivity, and Accuracy. 




In the category of Quality, representative comments 




include: "I felt many times the observer wrote things 


o 
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just to fill up space on the sheet--things that were 
•irrelevant to the observation," and ''Much too general. 
The system is too open ended." Lack of specificity and 
failure to provide meaningful suggestions and comments 
were areas of concern most frequently mentioned in this 
category . 

Observer Subjectivity was also criticized. Some 
students commented that the field note feedback was 
"Mostly opinions. The feedback is based on how someones' 
own teaching techniques (are),'* and "The only problem is 
that the supervisor is judging me by personal standards 
and ideas of what 'good' teaching is." The observers 
were not always perceived as expert teachers and, when 
this was the case, their comments and suggestions were 
not taken seriously. This position was supported by 
other students ' comments which questioned the accuracy of 
the feedback they received. 

When asked what they liked about the computer 
observation system feedback, student comments fell into 
five categories: Clarity, Specificity, Breadth, 
Efficiency, and Personal Instructional Information. 
Examples of each include the following comments, 
respectively: "It broke everything down into small 
categories, which gave a clearer picture of how I was 
Hoing"; "It's concrete on the percentages. It evaluated 
things I don't really think of"; "This is a very 
comprehensive feedback system and allows the instructor 
to evaluate many things, things that the observer may not 



have otherwise had time to comment on"; »*It was an 
efficient way of covering specific parts of the 
observation of the student teaching process. It helped 
to put things in perspective for me the student so far as 
the types of things I might improve to be a more 
effective teacher"; and "To see improvements from the 
beginning that can be used for comparisons." This last 
category, Personal Instructional Information, was by far 
the largest, with 18 comments related to this aspect of 
the computer-based feedback. 

Negative comments regarding the computer feedback 
fell into the categories of: Inaccurate, Confusing, and 
Distracting. Examples of each category include: "I did 
not think that the percentages of time I was actively 
instructing were a fair estimate of what was actually 
taking place. I thought I was activelly (sic) teaching 
more than the computer sometimes indicated"; "Too 
confusing, never understood clearly"; and "It was 
extremely annoying to have the printout run in class. 
This happened to me once in the fall and once in the 
spring . " 

Thus, the students liked the computer observation 
system because it was specific, comprehensive, and able 
to objectively gauge an individual's improvement over 
time. However, they still valued the descriptive 
information provided by written accounts of the observed 
lesson. The computers gave terse, objective performance 



89 

data on specific teaching behaviors. Some students found 
this form of feedback somewhat more difficult to 
interpret . 

When asked if their feelings regarding the computer- 
based feedback changed as a result of the ALT training, 
the majority of students answered in a positive 
direction, with 13 students agreeing that the training 
was helpful in understanding and applying the 
information. At the same time, five students indicated 
that they did not find the ALT training personally 
helpf ul . 

Finally, students were asked to suggest what they 
perceived the most beneficial type of feedback to be. 
Fourteen answered that field note feedback alone would be 
best, although 10 of those 14 added a caveat that more 
discussion was necessary to make this form of feedback 
useful. Three answered that computer feedback alone 
would be best, while 13 felt that a combination of f^eld 
notes and computer-based feedback would be optimal. 
Finally, three individuals mentioned that they would 
prefer to have their supervising teacher, i.e., the 
classroom teacher, evaluate their instructional 
performance . 

In conclusion, while students were more positive in 
their opinions regarding the field note feedback, the 
reasons cited were not closely related to the improvement 
of instructional behavior, i.e., they liked the field 
notes because they were '^related*' to what actually 
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happened during the observation. Some perceived the 
computer feedback as "hard da^a," unable to take 
classroom environmental variables into account. The 
positive comments made regarding the computer-based 
feedback , however, stressed its specificity, its 
comprehensiveness, and its ability to objectively measure 
change over time regarding the use of more effective 
teaching strategies over time. The observational data 
clearly show that this was the form of feedback which 
caused significant improvement in actual teaching behavior. 

In conclusion, students indicated that the most 
helpful form of performance feedback would be some 
combination of field note and computer feedback. Field 
notes provide important contextual information while the 
objective feedback caused significant positive changes ^n 
the use of more effective teaching practices by the 
preservice teachers involved in this study. 
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DISCUSSION 

This study was conducted with two groups of students 
over three periods, or phases • During the baseline 
phase, both groups received field note feedback from the 
observers regarding their classroom teaching behavior. 
No significant differences were observed between the 
groups on the dependent variables (e,g,, levels of 
student engagement , active instruction, passive 
instruction and transition). In other words, one would 
be unable to discriminate between the groups based upon 
their observed performance during the baseline phase. 
For purposes of discussion, observational data results 
will be addressed first, followed by checklist item and 
finally, attitude scale results. 

Following the first training session when Group A 
received the ALT training /feedback , differences in 
performance levels on the dependent variables were 
observed: students in Group A attained a significantly 
higher level of active instruction than students in Group 
B, even though Group B had a mean increase of 19 per 
cent. This difference indicates that the ALT 
training/feedback altered the students' use of specific 
instructional behaviors more than the alternate, e.g,, 
humanistic classroom management, training combined with 
traditional field note feedback. As hypothesized, when 
Group B received the ALT training and feedback, i,e,, 
during the second intervention phase , their active 
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instruction level also showed significant gains, becoming 
highly similar to the observed means of Group A. No 
differences were observed between the groups in the 
category of active instruction during this period. 

This same pattern held true for observed levels of 
passive instruction. After obtaining highly similar 
baseline means, both groups reduced their use of this 
instructional strategy during the first intervention 
phase, but Group A's reduction was much sharper (23 to 3 
per cent) than that of Group B (26 to 19 per cent). 
Again, the ALT training and computer feedback appeared to 
affect teacher trainee behavior more strongly than the 
alternate training and field note feedback. The levels 
of passive in5>truction observed for both groups became 
nearly identical during the second intervention phase, 
after Group B had received the ALT training. 

This same pattern was again observed in the category 
of transition. Group A had a mean transition level which 
was significantly lo,;er than Group B's mean following 
ALT training and feedback. Group B showed a significant 
reduction in their observed transition score during 
the second intervention phase, after they received the 
ALT training/feedback. Unexpectedly, Group A's level 
rose slightly between the first and second intervention 
phases and that increase, combined with Group B's marked 
reduction, created a significant difference between the 
groups during the second intervention phase, when it was 
expected that the group means would be more homogeneous. 
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As this study controlled for effects due to 
subjects' history, maturation, and skill, the observed 
results indicate that the ALT training and feedback was a 
more effective strategy than the use of primarily field 
note feedback in influencing students to alter observed 
instructional behavior specifically related to Academic 
Learning Time categories. 

The positive influence of the combined ALT training 
and feedback was also observed when each groups' use of 
effective teaching behaviors was examined over the three 
phases of the study. Observational data indicate that 
Group A increased their level of active instruction an 
average of 28 percentage points between baseline and the 
first intervention phase, following the research-based 
training. Students in this group had an average of 95 
per cent active instruction during the first intervention 
phase. Because the observed group mean was so high, some 
students almost certainly experienced a ceiling effect, 
indicating that the observed improvement was distributed 
among all students in the group. This high level of 
active instruction was also maintained through the final 
phase of the study. 

Concomitantly, the passive instruction and 
transition levels of Group A students showed a 
significant reduction during the first intervention 
phase, remaining low throughout the study. All of the5^e 
changes indicate that ALT training, in conjunction with 
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related computer-based obs arvational feedback, caused 
significant improvement in the way these students 
structured their teaching • 

In addition to the students in Group A increasing 
their use of more effective teaching strategies, 
beneficial effects were also observed on classroom 
student engagement levels during the first intervention 
phase. Active student eng-gement showed a significant 
increase, while passive student engagement levels 
declined. Since these cojstitute an ipsative measure, 
i.e., if one increases ^: a other decreases, this change 
was expected. However, in addition to statistical 
significance, the observed ' ^rease in active student 
engagement shows both practical and theoretical 
significance, for this variable has consistently been 
related with increased student achievement. 

The highly positive results observed in Group A's 
teaching behaviors during the first intervention phase 
are tempered somewhat by the results observed in the 
second intervention phase. During this time, student use 
of active instruction decreased eight percentage points, 
while passive instruction increased four percent and the 
level of transition increased by three percentage points. 
Student engagement levels changed little however, having 
a combined loss of only one per cent. 

While the reasons for this reduction, albeit slight, 
in the use of more effective teaching behaviors might be 
many, four seem most plausible. First, the "booster*' 
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training provided students in Group A prior to the second 
intervention phase may not have been suffici 
to maintain previous levels. Second, the change of field 
placement may have had an influence. The second 
intervention phase of the study was conducted in urban 
classrooms, an environment highly unlike previous 
plsceraents. This difference may have accounted for some 
of the observed reductions in the dependent variables. 
Third, a ceiling effect may have been a factor. Students 
in Group A obtained a mean level of 95 per cent in this 
category, with a standard deviation of 5,2 percentage 
points. In other words, some students were actively 
instructing the entire observational period. This is an 
unrealistic and perha: even undesirable level to 
maintain, and the reduction may have reflected a 
naturally occurring correction effect. Finally, students 
in Group B demonstrated such a dramatic reduction in 
their transition rate during this same phase and this 
reduction, combined with Group A's slight increase, 
caused the observed difference between the groups. This 
effect could also be interpreted as an artifact of the 
potency of the ALT training/feedback paradigm, 

any rate, the observed changes were minimal and 
may in actuality indicate a more realistic instructional 
profile than the trainees' consistently obtainiiig optimal 
levels. If the majority of public school teachers 
achieved active instruction rates in the high 80 's and 
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student engagement rates in the low 90 per cent range, 
resulting in the predicted gains in student performance, 
the present clamor for accountability and imposed 
teacher regulations would be quieted. 

Observational data regarding classroom student 
engagement rates during the second intervention phase for 
both groups were disappointing. While these rates 
approached, they failed to reach the pre-established 
level of significance, indicating a lack of 
correspondence between improved teaching performance and 
subsequent effects on student on-task behavior. From the 
available data, a one-to-one relationship between these 
two events cannot be extrapolated. It would probably be 
most accurate to say that the use of effective teaching 
strategies influences, but does not. control, increased 
student task engagement. 

Because students in Group B received information 
unrelated to research-based teaching behaviors or to the 
computer observation system, no changes were expected in 
their observed levels on the dependent variables during 
the first intervention phase. In actuality. Group B 
students showed positive change on all three teacher 
behavior variables. Their level of active instruction 
increased 10 percentage points, while their passive 
instruction and transition levels decreased by six and 
three per cent, respectively. However, none of these 
changes were significant. 

It is unclear whether students in Group B would have 
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continued to gradually improve in their observed use of 
more effective teaching behaviors over the final phase of 
the study* This question could have been clarified by 
having a third group with an extended baseline phase 
in order to observe whether the trend toward increased 
use of effective teaching behaviors was repeated, and 
whether or not these behaviors were maintained over time» 
Data from this study, however, indicate that while the 
changes in teacher instructional behavior were positive, 
they were not nearly as robust as those observed in Group 
A during the same period. Neither the humanistic 
classroom management training nor the field note feedback 
significantly affected Group B studen+'.s' instructional 
behavior during this first intervention phase. 

During the second intervention phase, after Group B 
had received the ALT training and feedback, their mean 
active instruction level rose significantly over their 
baseline mean, while their passive instruction rate 
dropped significantly (from 20 to 5 per cent), 
replicating the effects observed in Group A. 
Additionally, the amount of time spent in transitional 
activities showed a significant reduction, indicating 
that the students were more aware of research findings 
relating to this instructional variable. 

Again, increases observed in the use of more 
effective teaching behaviors did not significantly affect 
the student engagement levels observed for Group 3 during 



the second intervention phase. Again, ceiling effects 
may have influenced the results, since the combined, 
i.e-, active and passive, student engagement level for 
students in this group ranged from 88 to 92 per cent over 
the course of the study. This lack of statistically 
significant improvement, combined with the inconclusive 
nature of the between groups difference, tends to confirm 
the imprecise relationship between the increased use of 
effective teaching behaviors and classroom student 
engagement levels. Good teaching does not always assure 
student engagement. However, the converse, i.e., that 
student engagement tends to occur in the presence of 
effective teaching practices, was clearly substantiated 
by the results of this study. 

The observational data obtained on the teaching 
behavior of students involved in this study provides a 
strong case that the combination of ALT training and 
computer-based observational feedback has a more 
significant effect in altering students use of specific 
teaching behaviors than traditional feedback procedures. 
Although Group B's observed levels on the dependent 
variables increased from baseline to the first 
intervention phase, the increase was slight, whereas 
G"^oup A srudents showed significant improvement. Also, 
each group showed significant improvement in their use of 
more effective teaching behaviors from their individual 
baseline periods to the appropriate intervention phase. 
These results indicate that, while traditional feedback 



in conjunction with additional classroom experience may 
cause students zo show minimal improvement, research- 
based information in combination with objective feedback 
directly related to that information produces far 
stronger changes in observed behavior. 

These accumulated results are viewed as evidence 
that students are capable of understanding and applying 
research-based information early in their professional 
career development. They are able to interpret objective 
performance feedback and use the information to shape 
their classroom behavior in a positive direction. These 
changes in teacher behavior theoretically serve to create 
a classroom learning environment which is more conducive 
to student academic achievement . 

The checklist items were also used to evaluate 
student growth in implementing more qualitative teaching 
behaviors. Because of its limited range (each item was 
rated on a 3-point scale), significant differences 
between the groups were not anticipated, although it was 
expected that the groups would show improvement in their 
mean ratings following the ALT training. 

For the most part, this was indeed the observed 
effect. Only four items significantly differentiated 
between the groups following the first intervention 
phase. These items related to the teacher trainees' 
conveying to pupils the importance of the instructional 
activity and the quality of student performance. Also, 



100 

items addressing lesson and time organization showed 
significant differences between the groups in favor of 
Group A during the first intervention phase. During the 
second intervention phase, i.e,, after both groups had 
received the ALT training, only one significant 
difference was observed between the groups, and it 
involved the pacing of instruction. 

However, when the mean ratings on the checklist 
items for each group were examined individually, 
differences became much mere obvious: Group A students 
significantly increased their scores on 15 of the 20 
checklist items during the first intervention phase. 
Furthermore, while the number of items selected for 
analysis for this study was restricted to 20, 
observers were directed to rate the students on any of 68 
items judged appropriate for a given student and/or 
situation. From the baseline through the first 
intervention phase, students in Group A showed 
significant improvement on a total of 32 items, a change 
that seems highly unlikely to occur as a result of chance 
improvement alone. Thus, the ALT training appears to 
have influenced positive change on almost half of 
the total items in the checklist for students in Group A. 

It was hypothesized that students in Group B would 
show little change in their mean checklist ratings 
between the baseline and first intervention phase, then 
show greater improvement between the first and second 
intervention phases. Tne data indicate that, in the first 
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comparison period, Group B students showed significant 
iinprovement on only two of the 20 items selected for 
analysis and on only four out of the total 68 items. 
Between the first and second intervention phases, 
significant improvement was observed on five of the 20 
items arxd on eight of the total checklist items. Thus, 
while students in Group B more than doubled the number of 
items showing highly significant improvement, the effect 
was not as large as anticipated. 

These findings suggest that after teachers received 
the ALT training and feedback from the checklist data 
source, they became more aware of the dynamics involved 
in teaching, i.e., lesson flow, alerting students to both 
the intent of the instructional activity and the value of 
their contribution to it. However, any conclusions drawn 
from the available data must be viewed tentatively 
because of the limited number and magnitude of the 
differences and the restricted evaluative range of the 
checklist rating scale. 

The attitude scale was initially administered to 
both groups at the end of the first intervention phase, 
after students in Group A had received both the ALT 
training and three computer-based performance 
evaluations; at that point, students in Group B had 
received the alternate training and field note feedback 
exclusively. The attitude scale was administered again 
at the 'onclusion of the study, after both groups had 
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received the ALT training. 

Both groups expressed highly positive opinions 
regarding the field note feedback throughout the study* 
Students, in general, indicated that the supervisors were 
skillful in providing field note feedback, that field 
note feedback was helpful ir improving their teaching 
skills, and that this information provided them with 
ideas of things they needed to do to improve their 
teaching. They also stated that they felt comfortable 
with this form of performance evaluation and that field 
note feedback should be retained as part of the field 
supervision program. Two reasons most frequently cited 
in their comments were that this type of feedback is more 
situation-specific and also is more personally relevant 
than the computer-based feedback. 

On the first administration of the attitude scale, 
students in Groups A and B diverged significantly in 
their responses to the computer observation system items. 
Group A students indicated that they understood the 
observational categories significantly more often than 
Group B students. Since the sole experimental 
manipulation regarding this item involved the 
differential training received by rhc students, it can be 
assumed that the ALT training influenced the more 
positive response to this item. Students in Group A also 
indicated thai the computer feedback helped them improve 
their teaching significantly more often than Group B 
students . 
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None of the students reported feeling threatened by 
the microcomputer observation system • However, when 
asked whether this form of feedback should be retained as 
part of the field supervision program on the first 
attitude scale administration, Group B students 
answered in a negative direction, w;:ereas Group A 
students felt significantly more positive about retaining 
it. This difference between the groups disappeared after 
the second ALT training. These results indicate that, 
when students understood and were trained to use the 
information to improve their teaching, they perceived 
it in a more positive manner. 

Results also indicated that the groups differed 
significantly in their perceptions of the checklist item 
feedback following the first intervention phase. 
Students in both groups indicated that the observers' 
ratings of checkl ist i tems were helpf ul in improving their 
teaching skills. However, students in Group B felt that 
the checklist feedback system was too confusing to be 
useful, a seemingly contradictory response. Some 
students in this group noted that the 3~point scale was 
not sensitive enouf^h, i.e., that it did not specify what 
was wrong or how it could be improved. Perhaps this 
criticism influenced them to answer the second item in 
a L.ore negative direction. 

Results from the initial administration of the 
attitude scale indicated that students who had received 
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the ALT training expressed significantly more positive 
attitudes about retaining the checklist feedback system 
than students who had received the alternate training. 
This ^lifference disappeared after both groups had 
received the ALT training and computer-based feedback. 

To summarize, student responses to the attitude 
scale items appear to indicate that thorough training is 
essential to the acceptance and understanding of the 
computer-based observational feedback. When students 
lack an adequate knowledge base regarding the 
quantitative feedback, they view it with some annoyance 
and little understanding. More importantly, they are 
unable to see its contribution to their professional 
development. However, when they understand the 
relationship between the observational data and their 
teaching performance, they are able to use it in a way 
that significantly increases their use of more effective 
teaching behtwiors. At the same time, students still 
value field note feedback provided by competent 
observers. This information allows them to place their 
lessons in context, something missing in the more 
quantitative performance evaluation. Both groups 
unanimously indicated that both forms of feedback should 
be utilized in the undergraduate field supervision 
program after the ALT training and feedback had been 
provided. 

While the results obtained in this study clearly 
indicate that training is essential in the application of 



specific knowledge to improve individual teaching skills 
and to student acceptance of a novel yet effective form 
of performance feedback, the study could have been 
improved by incorporating certain changes. One 
would have been to increase the number of observations. 
Since this study was conducted within the constraints of 
an existing preservice teacher education program, this 
alternative was impossible. It has been noted repeatedly 
that naturalistic observations are the greatest 
expenditure in field research (Johnston & Bolstad, 1978). 
This factor certrinly contributed to the relatively small 
number of observations upon which this study was based. 
If the basic design were to be replicated with a greater 
number of student observations, time series analysis 
could be utilized to further illuminate the relative 
effects of experience, differential feedbacK, placement, 
and training . 

Second, a more accurate performance evaluation may 
have been obtained using a less obtrusive observation 
technique. Because these observations were conducted in 
naturalistic settings, the situation did not allow 
unobtrusive observation conditions, e.g., one-way mirrors 
where teacher trainees were unaware they were being 
observed. The intent of this study was to inform these 
trainees of more effective teaching techniques as 
identified in the research literature, and to provide 
feedback regarding their use in natural situations. This 
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goal was accomplished, so perhaps covert observations 
would not have been an appropriate technique to use 
during this phase of their professional development. 

Third, more field reliabilities could have been 
obtained. Again, cost and time factors proved 
prohibitive within the constraints of the present study. 
However, were the study to be replicated, this would 
represent a significant improvement. Indeed, replication 
is strongly suggested, for it is unwise to accept the 
observed relationships as real on the basis of only one 
study, no matter how significant tue results (Borg & 
Gall, 1983). 

Finally, the checklist and attitude scale items did 
not undergo sufficient field testing, so the issues of 
rating reliability and item validity need to be examined. 
These scales were primarily intended to gauge qualitative 
performance and attitude change in the students as a 
function of the different experimental treatments. All 
reported results should be viewed with this purpose 
in mind. Future studies can and should determine 
individual item reliability and scale validity. 
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CONCLUSION AND IMPLICATIONS 
The results of this study clearly demonstrate the 
potency of a specific training program in conjunction 
with an objective feedback system for signif icartly 
increasing specific preservice teaching behaviors. The 
present data both support and extend effects observed 
with inservice special education teachers (Semmel et al., 
1976; Rieth & Frick, 1978). Providing teachers with 
research-based information and objective performance 
feedback results in significant improvements in their use 
of more effective teaching behaviors. This study also 
demonstrated that high levels of student engagement were 
dependent upon teachers' displaying higher levels of 
active instruction and lower levels of passive 
instruction and transition. However, a direct 
relationship between increased levels of ALT teaching 
behrviors and student engagement was not found. 

Future studies should be undertaken to determine if 
the effects observed in this study are replicated in 
different settings. This would also help to validate the 
observation system as well as thv^ checklist and attitude 
scale items. Expanding the ratin.'t scale for the 
checklist items should be considered. Otherwise, ?t 
might be omitted from further investigation, proving too 
subjective an instrument to be an effective evaluation 
tool . 

The observation system could be expanded to include 
other instructional procedures which have been 
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empirically related to increased student academic 
achievement • One example might involve i nc 1 lid i n 
information and observational feedback about students' 
application of specific behavior management techniques as 
well as their effect on pupil behavior. Another might 
examine teachers' lesson structure, e.g., review 
procedures, clarification/explanation period, 
demonstration, provision of successful practice, etc. 

Finally, normative data sLo Id be collected, both on 
preservice teacher behavior rates as well as current 
teachers who have been identified by their peers as 
"master" teachers at p^l educational levels. This 
information would p- . ide ranges of optimal criterion 
levels for future observational feedback. The use of 
such an expert systems approach, objectively examining 
what good teachers do as they go about their teaching as 
well as viat students do in response to these actions, 
would go a long way toward obtaining various profiles of 
truly effective teachers. Once a series of profiles had 
been established, it could be empirically validated and 
serve as a model for preservice teache^^s. 

Student participants involved in this study were 
very interested in becoming more effective teachers. 
Further clarification of what constitutes good eaching, 
the provision of specific information and consistent, 
objective feedback to teachei trainees would add great 
credibility to existing teacher education programs. 



Teaching is more than an art, it is the thoughtful 
evecution of a number of specific behaviors, behavior 
which can be mastered by prospective teachers. This 
study has demonstrated one way this might be 
accomplished . 
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Appendix A: Standard Field Note Feedback Form 
INSTRUCTIONAL OBSERVATION 



Student Observed Sally Student 

Subject/Grade level History/Junior High 
Focus for Observation general 



Sally was working with a small group of three boys and two girls, 
reviewing facts from a previous history lesson, Sally would ask each 
a question in turn. If that student could not answer it, others would 
raise their hands to answer. This procedure was followed for about 
20 minutes, then Sally provided a 10-minute explanation of some home- 
work she wanted the students to complete for the next. day, 

+ You maintained good eye contact with the students who were answering 
the questions. 

+ Touching Jimmy on the knee when his attention wandered half way 
through the lesson was effective in "bringing him back" into the 
group, 

- When one student was answering a question, I noticed that the others, 
especially the ones farthest away from you, were not paying attention 
ro the answer. Perhaps if you had asked the questions randomly, the 
students would have paid closer attention, thinking that they might 
be called on next* 

- The homework explanation was not very clear • You could have brought 
in some examples of what you were describing, to show the students 
exactly what ^^ou wanted, I noticed many students doing other work 
while you were describing the assignment, and wondered if they were 
hearing what you wanted, and if they would be able to do the work. 



Date January 20, 1986 

Time 1:30 tn 2:00 D.rt. 



Observed by Observer 13 



Student Copy (white) 

! P^y^ Jing Teacher Copy (yellow) , 
I cKJL Jy Supervisor Copy (pink) ^ ^ 6 



Appendix B. Training Material showing Relationship 
Between Various Levels of Teacher Behavior and Subsequent Effects on Student Perf.raance 



^er iod: 

Student Number: ' 
Mame: 

Soeciat Education Status: 
>»umber of Observations (in 
Anounc of Observation Tims 

3'?5;erw'ation code SUM'^ARY 



s umma ry ) : 
(in s umma ry ) 



•JUN 30 5 1982 to SEP Oit 1985 

MAJOR 
7 

"227 ■m'i m 1 nut et s) 



JROUP STRUCTURE 
NULL 

JHOLE tiKOUP RilSPONSIB' TY 
PARTIAL GROUP RE3PCNS LITY 

PART 5RP re: ons ^:th ^ootl monitoring 

I.MOIUIOUAL ^tSPO^JSIaILITY ^"^^''^^i^^ 

EDUCATIONAL ACTIVITY 
NULL 

ACTIVE INSTRij^- IQ^J 
TRANSITION 

TEACHER INSTRUCTIONAL 3Z■-i^\llOR 
NULL 

STRUCTURING-^OIRECTIVJO 

EX'»LAMATI0N+TUESTICNTM3-PLANNE0 
EXPL Af ATION>OUEST rCNI 'JG-NE'^O 
EVALUATIVE FEE03ACK 
TASK ENGAGEMENT FEEDBACK 
BEHAVIORAL FEEDBACK 



BEHA'VTORS CTAR.-JET GROUP) 



STUOENT(S) 

m, 

ENGA Oeo-ACTi Vf^^^ 
ENGA*«;ED-PASS*rvE J 
^GAGED-/ICT-^E 
NON -ENGAGEO-PASSI VE 

3TU0ENT{S) BEHAVIORS (MOMlTORrr, qkOUP) 
ENGAGED-ACTIVE 



PERCENT 
. 5X 
56. 7X 
22 ,2% 
9. OX 
11.9% 

PERCENT 
.2% 

. 3;!: 

PERCENT 
.2% 

2, or. 
79, ax 

8 ,0% 
6.5X 
2.3% 
1.2% 



°ERCENT 




PERCENT 
100. OX 



MINUTES 
• 6) 
134 .5) 
52. &) 
21. 4) 
28.2) 

MINUTES 

23^.1 ) 
.7) 



MINUTES 
.^) 
4.8) 
189.3) 
l'='.C) 
15.4) 
5.4 ) 
3.0 ) 



MINUTES 
.5) 
228 .1 ) 
3.9) 
3.4) 
1 .'^) 



MINUTES 
(■ 21.1) 



3TEEL/IMS (OCooyri^ht 1985 CITH/ Inoiona UnWer.ity 
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Appendix Cr Sample Computer Observation Data Feedback 



STEEL 
Observation Sys+em 

Report 
Date: 89/24/35 
Observer: 87 
Subject: 145 



Observation Codes 
Total Time: 0: 8: 3 



GROUP STRUCTURE 
PART 6RP W/MON :100. 0'/. 



ED ACTIUITV 
ACTIUE n^STP : 49.4'/. 
PASSIUE IHSTR : 50, 6"; 



TCHR INSTR BEH 

OBSERi; MONITOR : 20, 3:^^ 

STRUCTURE/DIPECT: 2. 3^: 

EXP/QUEST-PLhN : 41. 6:^^ 

EI-^p/QUEST-f^EED : 7c S^; 

EUALUhTI'JE Fl-BK : 5, Z'4 

TASK ENGAGE FDEK: 14. Z'4 

EEHAUIORAL FDBK : 8. 4^: 



PR I STUDENT BEH 
EK'GAGED/ACTIUE : 43, 0'; 
E -'>GED ''PASS I UE : 49. 2^' 
K... -ENGAGE/ ACT : 3.?': 
NON-ENGAGE/PhSS r 3. 9': 



SEC STUDENT BEH 
ENGAGED. 'ACTK'E : 55. 9:; 
ENGAGED/PASS I^E : 3^1. 

non-enghGE/hCT : 7. n:: 

NON-ENGAGE/PhSS : ^S. G^i 

Checklist Coder 
IMP OF CUPR CONT: HI 
DEG OF TSK-OPIEN: HI 
SEQ OF INSTR : HI 
USE OF DIFF MOC'E: h£b 
ACK OF PUP INPUT: HI 
B<HANCE PUP KNOM: HI 
.^GE-hPPPOP : HI 

LEG IB OF WRITING: MED 
HMhPE of CLS C''.'M: MEC' 

clhPif BEH e::pct: hi 

USE OF NON-UPB 0: HI 
ADAPT TO CHG SIT: meD 
ABLTV TO PEL'^rZ : HI 
POS INTER IJ-T'iPS: .lED 
DEG STU TSf: "<!£: MED 
LUL OF GRP Ph.RT : MED 
RELE" or PAPTICP: MED 
"tHpIETV PHPT nOD: MED 

End of -eport 
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STEEL 
Observation System 

Report 
Date: 03/17/86 
Observer: 14 
Subject: 169 



Obser '.nation Codes 
Total Time: 0: 4: 22 



GROUP STRUCTfiRE 
I'JHOLE GROUP :iCiO, 0^; 



ED ACTIUITV 
ACTI'E INSTR : 4"^, 1^. 
PASSIUE INSTR : 56. 9:^^ 

TCHR INSTR BEH 

OBSERU MONITOR : 49. 

STRUCTURE/DIRECT: 6.9^ 

E;JvnuEST-FLAN : l-^.S^. 

E>:P/nnEST-NEED : 3. 4": 

E'«ALUqTIUE FDBK : 4. A'; 

TASK ENGAGE FDBK: 3. V: 

BEHhVIOPAL FDBK : 0^ 



PR I STUDENT BEH 
ENGAGED/ACTIVE : 25, 2:^ 
ENGAGED -'PASSIUE : 64.5'/. 
NOH-ENGhGE/ACT : 6* 5^; 
NON-ENGAGE/PASS : 



SEC STUDENT BEH 
NULL : 100. Qy. 

Checklist Cod^- 
IMP OF PUP PEPF : Hi 
ORG OF PRESENT : HI 
PROU OF FEEC^BhCK: MED 
EDUC PELDJhNCE : HI 
AGE-hPPROP : HI 

STIMULUS-UhLUE : HI 
AI.IARE OF CLS D'.'M: HI 
ADhPT to CHG SIT: HI 
MOC'EPATE UijICE : MED 
DEG STU TSir OPIE: HI 
UhRIETV part MOD: MED 

End of Report 
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Appendix D 

Student Evaluation of Observation System 

1* The field-note feedback was helpful in improving my 
teaching skills 

2* The supervisors were skillful in providing field-note 
feedback, 

3, The computer was always used during the observations, 

4, I always received the computer summary from the 
observers , 

5, The computer-based feedback was helpful in improving 
my teaching skills. 

6, I clearly understand the various categories on the 
observation system • 

?• The supervisors clearly explained the computer 
summari es of my teaching behavi orr, • 

8, The checklist feedback (High-Medium-Low) was 
helpful in improving my teaching skills. 

9. The supervisors were skillful in explaining the 
checklist feedback • 

10. The computer observation feedback should be 
retained as part of the undergraduate field 
supervision program. 

11. The field note feedback should be retained 
as part of the undergraduate field 
supervi sion program . 

12. The checklist feedback system should be 
retained as part of the undergraduate field 
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supervision program. 

13. Being evaluated with the computer observation 
system is a threatening experience. 

14. The field note feedback doesn't provide me 
with clear ideas of v^hat i;pecific thin^^s I 
need to do to improve my teaching. 

15. I feel the checklist feedback system too 
confusing to be useful. 

16 . I feel comfortable wi th supervisors us ing 
the computer observation system to evaluate 
my teaching. 

17. I feel comfortable with supervisors using the 
field note method to evaluate my teaching. 



For the following items, feel free to include ctber 
comments on the back of the page if you need more space: 

IS. What things do you like about the field-note 
f eedback? 

20. What things do you dislike about the field-note 
feedback? 

21. What uhings do you like about the computer-based 
observation feedback system? 

22. What things do you dislike about the computer-based 
observation feedback system? 

23. In your estimation, what would be the most helpful 
type of feedback regarding your practice teaching 
performance? 
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24. Did your feelings regardin.^ the computer-based 
feedback change as a result of the training you 
received? In what way(s)? 
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